Redis 集群及将 Redis 作为 mysql 的缓存服务器实战
来源：互联网 发布：网络骑士是哪里人 编辑：程序博客网 时间：2024/05/29 18:27
一、Redis简介
redis 是一个高性能的 key-value 数据库。 redis 的出现,很大程度补偿了memcached 这类 keyvalue 存储的不足,在部分场合可以对关系数据库起到很好的补充作用。它提供了 Python,Ruby,Erlang,PHP 客户端,使用很方便。Redis 的所有数据都是保存在内存中,然后不定期的通过异步方式保存到磁盘上(这称为“半持久化模式”);也可以把每一次数据变化都写入到一个 appendonly file(aof)里面(这称为“全持久化模式”)。
二、Redis 安装
本次实验所用所有安装的rpm包及源码资源均已上传，需要的可以下载下来
地址：
redis.tar.gz.0
redis.tar.gz.1
需要将这两个都下载下来，并放到同一目录下，然后执行
cat redis.tar.gz.* | tar -zxv即可解开
1.下载安装
redis官网：https://redis.io/
下载：http://download.redis.io/releases/redis-4.0.2.tar.gz
[root@server2 ~]# yum install -y gcc[root@server2 ~]# wget http://download.redis.io/releases/redis-4.0.2.tar.gz[root@server2 ~]# tar -zxf redis-4.0.2.tar.gz[root@server2 ~]# cd redis-4.0.2[root@server2 redis-4.0.2]# make && make install[root@server2 redis-4.0.2]# ls /usr/local/bin/alien redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-sentinel redis-server这些可执行文件的作用如下:
redis-server: Redis 服务主程序。
redis-cli: Redis 客户端命令行工具,也可以用 telnet 来操作。
redis-benchmark: Redis 性能测试工具,用于测试读写性能。
redis-check-aof:检查 redis aof 文件完整性,aof 文件持久化记录服务器执行的所有写操作命令,用于还原数据。
redis-check-dump:检查 redis rdb 文件的完整性,rdb 就是快照存储, 即按照一定的策略周期性的将数据保存到磁盘,是默认的持久化方式。
redis-sentinel:redis-sentinel 是集群管理工具,主要负责主从切换。
2.配置并启动服务
[root@server2 redis-4.0.2]# cd utils/[root@server2 utils]# ./install_server.shWelcome to the redis service installerThis script will help you easily set up a running redis serverPlease select the redis port for this instance: [6379] Selecting default: 6379Please select the redis config file name [/etc/redis/6379.conf] Selected default - /etc/redis/6379.confPlease select the redis log file name [/var/log/redis_6379.log] Selected default - /var/log/redis_6379.logPlease select the data directory for this instance [/var/lib/redis/6379] Selected default - /var/lib/redis/6379Please select the redis executable path [/usr/local/bin/redis-server] Selected config:Port : 6379Config file : /etc/redis/6379.confLog file : /var/log/redis_6379.logData dir : /var/lib/redis/6379Executable : /usr/local/bin/redis-serverCli Executable : /usr/local/bin/redis-cliIs this ok? Then press ENTER to go on or Ctrl-C to abort.Copied /tmp/6379.conf => /etc/init.d/redis_6379Installing service...Successfully added to chkconfig!Successfully added to runlevels 345!Starting Redis server...Installation successful![root@server2 utils]# netstat -antlp |grep redistcp 0 0 127.0.0.1:6379 0.0.0.0:* LISTEN 4733/redis-server 1redis 主配置文件为/etc/redis/6379.conf
这里只在本机做测试，所以不做更改了，如果外部主机进行访问，需要设置bind 和port两个参数,默认监听本机所有网络接口，redis 服务的端口号是6379，可根据需要做相应修改
#bind 127.0.0.1bind 0.0.0.0#port 6379port 63803.Redis 客户端使用
[root@server2 utils]# redis-cli 127.0.0.1:6379> config get * 1) "dbfilename" 2) "dump.rdb" 3) "requirepass" 4) "" 5) "masterauth" 6) "" 7) "cluster-announce-ip" 8) ""上面说的可以通过修改配置文件来修改配置，另外还可以使用 CONFIG set 命令来修改配置,但重启服务后会读取 redis.conf 文件配置。
127.0.0.1:6379> CONFIG GET loglevel1) "loglevel"2) "notice"127.0.0.1:6379> CONFIG SET loglevel "debug"OK三、Redis集群
1.概述
- 节点自动发现
- slave->master 选举,集群容错
- 所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.
- 节点的fail是通过集群中超过半数的节点检测失效时才生效.
- 客户端与redis节点直连,不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可
redis-cluster选举:容错
- 领着选举过程是集群中所有master参与,如果半数以上master节点与master节点通信超过(cluster-node-timeout),认为当前master节点挂掉.
- 什么时候整个集群不可用(cluster_state:fail),当集群不可用时,所有对集群的操作做都不可用，收到((error) CLUSTERDOWN The cluster is down)错误
- 如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成进群的slot映射[0-16383]不完成时进入fail状态.
- 如果进群超过半数以上master挂掉，无论是否有slave集群进入fail状态.
2.实验环境
操作系统：redhat6.5
测试我们选择2台服务器，分别为：192.168.1.237，192.168.1.238.每分服务器有3个节点。
2.redis cluster安装
刚才我们已经安装好了，接下来创建节点
3.创建redis节点
[root@server2 ~]# /etc/init.d/redis_6379 stop[root@server2 ~]# cp redis-4.0.2/src/redis-trib.rb /usr/local/bin/ ##从刚才解压的文件里将redis-trib.rb拷贝到/usr/local/bin/目录下，或者将其添加进系统环境变量即可[root@server2 ~]# mkdir -p /usr/local/redis-cluster/3000{1..8} ##需要多少节点就添加多少个，我这里添加了8个[root@server2 ~]# cp /etc/redis/6379.conf /usr/local/redis-cluster/30001/[root@server2 ~]# vim /usr/local/redis-cluster/30001/redis.confdaemonize yes ##redis后台运行pidfile /usr/local/redis-cluster/30001/redis.pidlogfile /usr/local/redis-cluster/30001/redis.logport 30001dir /usr/local/redis-cluster/30001/cluster-enabled yes ##开启集群cluster-config-file cluster.conf ##集群的配置，配置文件cluster.conf首次启动自动生成，分别在对应的3000*目录下cluster-node-timeout 10000 ##请求超时appendonly yes ##aof日志开启 有需要就开启，它会每次写操作都记录一条日志[root@server2 ~]# sed 's/30001/30002/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30002/redis.conf[root@server2 ~]# sed 's/30001/30003/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30003/redis.conf[root@server2 ~]# sed 's/30001/30004/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30004/redis.conf[root@server2 ~]# sed 's/30001/30005/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30005/redis.conf[root@server2 ~]# sed 's/30001/30006/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30006/redis.conf[root@server2 ~]# sed 's/30001/30007/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30007/redis.conf[root@server2 ~]# sed 's/30001/30008/g' /usr/local/redis-cluster/30001/redis.conf >/usr/local/redis-cluster/30008/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30001/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30002/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30003/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30004/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30005/redis.conf[root@server2 ~]# redis-server /usr/local/redis-cluster/30006/redis.conf##这里先起六个节点，另外两台暂时不起[root@server2 ~]# ps -ef | grep 3000*root 4797 1 0 20:46 ? 00:00:00 redis-server *:30001 [cluster] root 4802 1 0 20:46 ? 00:00:00 redis-server *:30002 [cluster] root 4807 1 0 20:46 ? 00:00:00 redis-server *:30003 [cluster] root 4812 1 0 20:46 ? 00:00:00 redis-server *:30004 [cluster] root 4817 1 0 20:46 ? 00:00:00 redis-server *:30005 [cluster] root 4822 1 0 20:46 ? 00:00:00 redis-server *:30006 [cluster] root 4827 1018 0 20:49 pts/0 00:00:00 grep 3000*4.创建集群
官方提供了一个工具：redis-trib.rb(/usr/local/redis-3.2.1/src/redis-trib.rb) ,它是用ruby写的一个程序,所以我们需要安装ruby
[root@server2 ~]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/rubygems-1.3.7-5.el6.noarch.rpm[root@server2 ~]# yum install rubygems-1.3.7-5.el6.noarch.rpm -y[root@server2 ~]# gem install redis -v 3.3.5 ##需要联网，等一会就好了[root@server2 ~]# gem list*** LOCAL GEMS ***redis (3.3.5)[root@server2 ~]# redis-trib.rb create --replicas 1 127.0.0.1:30001 127.0.0.1:30002 127.0.0.1:30003 127.0.0.1:30004 127.0.0.1:30005 127.0.0.1:30006##创建集群>>> Creating cluster>>> Performing hash slots allocation on 6 nodes...Using 3 masters:127.0.0.1:30001127.0.0.1:30002127.0.0.1:30003Adding replica 127.0.0.1:30004 to 127.0.0.1:30001Adding replica 127.0.0.1:30005 to 127.0.0.1:30002Adding replica 127.0.0.1:30006 to 127.0.0.1:30003M: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots:0-5460 (5461 slots) masterM: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots:5461-10922 (5462 slots) masterM: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) masterS: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 replicates 7d285e51748c1683599dd31c6e27caf806867e2fS: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 replicates 8504c51e42a572219cc95fb3510d8156ed274a69S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6Can I set the above configuration? (type 'yes' to accept): yes>>> Nodes configuration updated>>> Assign a different config epoch to each node>>> Sending CLUSTER MEET messages to join the clusterWaiting for the cluster to join..>>> Performing Cluster Check (using node 127.0.0.1:30001)M: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots:0-5460 (5461 slots) master 1 additional replica(s)S: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots: (0 slots) slave replicates 8504c51e42a572219cc95fb3510d8156ed274a69M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6S: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots: (0 slots) slave replicates 7d285e51748c1683599dd31c6e27caf806867e2fM: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots:5461-10922 (5462 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.解释下参数含义
–replicas 1 表示自动为每一个master节点分配一个slave节点，上面有6个节点，程序会按照一定规则生成 3个master（主）3个slave(从)
如果不在同一台服务器上创建，防火墙一定要开放监听的端口，否则会创建失败。后面的ip和端口写对应的ip和端口就行了
需要注意的是在Can I set the above configuration? (type ‘yes’ to accept): yes 输入yes后提示 Waiting for the cluster to join……….
这个时候不要一直等，看提示Sending Cluster Meet Message to join the Cluster.
这里就需要去另一台服务器上做以下操作
redis-cli -c -p 7000* 分别进入redis各节点的客户端命令窗口， 依次输入 cluster meet 172.25.27.2 7000*
5.测试
1.无中心化测试
[root@server2 ~]# redis-trib.rb info 127.0.0.1:30001127.0.0.1:30001 (7d285e51...) -> 0 keys | 5461 slots | 1 slaves.127.0.0.1:30003 (42d80140...) -> 0 keys | 5461 slots | 1 slaves.127.0.0.1:30002 (8504c51e...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@server2 ~]# redis-trib.rb info 127.0.0.1:30004127.0.0.1:30001 (7d285e51...) -> 0 keys | 5461 slots | 1 slaves.127.0.0.1:30003 (42d80140...) -> 0 keys | 5461 slots | 1 slaves.127.0.0.1:30002 (8504c51e...) -> 0 keys | 5462 slots | 1 slaves.[OK] 0 keys in 3 masters.0.00 keys per slot on average.[root@server2 ~]# redis-trib.rb check 127.0.0.1:30005>>> Performing Cluster Check (using node 127.0.0.1:30005)S: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots: (0 slots) slave replicates 8504c51e42a572219cc95fb3510d8156ed274a69M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6M: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots:5461-10922 (5462 slots) master 1 additional replica(s)S: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots: (0 slots) slave replicates 7d285e51748c1683599dd31c6e27caf806867e2fM: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots:0-5460 (5461 slots) master 1 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.可以看到30001-3是master，30004-5为slave，并且30004->30001,30005->30002,30006->30003,访问任何一个节点均可以取到集群的信息，即使slave也可以，也可以通过任何一个节点进行读写操作
[root@server2 ~]# redis-cli -c -p 30005 ##-c表示支持集群模式，不加-c是不可以对集群进行操作的127.0.0.1:30005> get name-> Redirected to slot [5798] located at 127.0.0.1:30002(nil)127.0.0.1:30002> get name(nil)127.0.0.1:30002> set name mynameOK127.0.0.1:30002> get name"myname"127.0.0.1:30002> exit[root@server2 ~]# redis-cli -c -p 30004127.0.0.1:30004> get name-> Redirected to slot [5798] located at 127.0.0.1:30002"myname"127.0.0.1:30002>exit[root@server2 ~]# redis-cli -c -p 30001127.0.0.1:30001> get name-> Redirected to slot [5798] located at 127.0.0.1:30002"myname"127.0.0.1:30002>2.模拟故障测试
[root@server2 ~]# redis-cli -p 30001 shutdown ##master30001挂掉[root@server2 ~]# redis-trib.rb check 127.0.0.1:30005>>> Performing Cluster Check (using node 127.0.0.1:30005)S: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots: (0 slots) slave replicates 8504c51e42a572219cc95fb3510d8156ed274a69M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6M: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots:5461-10922 (5462 slots) master 1 additional replica(s)M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 ##30001的slave30004自动提升为master，这个时候集群还是正常工作的 slots:0-5460 (5461 slots) master 0 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.##测试能否正常工作[root@server2 ~]# redis-cli -c -p 30006127.0.0.1:30006> get name-> Redirected to slot [5798] located at 127.0.0.1:30002"myname"127.0.0.1:30002> set name qweOK127.0.0.1:30002> get name"qwe"127.0.0.1:30002> exit[root@server2 ~]# redis-cli -c -p 30002127.0.0.1:30002> get name"qwe"127.0.0.1:30002> shutdown ##在挂掉一台master30002not connected> exit[root@server2 ~]# redis-trib.rb check 127.0.0.1:30005>>> Performing Cluster Check (using node 127.0.0.1:30005)M: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 ##30005提升为新的master slots:5461-10922 (5462 slots) master 0 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:0-5460 (5461 slots) master 0 additional replica(s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.此时集群有半数以上（>=3）节点活着，并且没有挂掉一对主从，集群可以正常工作的，即使挂掉30003，集群仍可工作，但是如果挂掉30004或30005，则集群不能正常工作，为体现测是效果，我们将30001启动，启动即自动加入集群
[root@server2 ~]# redis-server /usr/local/redis-cluster/30001/redis.conf [root@server2 ~]# redis-trib.rb check 127.0.0.1:30005>>> Performing Cluster Check (using node 127.0.0.1:30005)M: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots:5461-10922 (5462 slots) master 0 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:0-5460 (5461 slots) master 1 additional replica(s)S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.此时此时挂掉30005
[root@server2 ~]# redis-cli -c -p 30006 get name"qwe"[root@server2 ~]# redis-cli -p 30005 shutdown[root@server2 ~]# redis-cli -c -p 30006 get nameCould not connect to Redis at 127.0.0.1:30005: Connection refusedCould not connect to Redis at 127.0.0.1:30005: Connection refused[root@server2 ~]# redis-cli -c -p 30001 get name(error) CLUSTERDOWN The cluster is down[root@server2 ~]# redis-cli -c -p 30003 get name(error) CLUSTERDOWN The cluster is down[root@server2 ~]# redis-cli -c -p 30004 get name(error) CLUSTERDOWN The cluster is down[root@server2 ~]# redis-cli -c -p 30006 get name(error) CLUSTERDOWN The cluster is down[root@server2 ~]# redis-trib.rb check 127.0.0.1:30001>>> Performing Cluster Check (using node 127.0.0.1:30001)S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6dM: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:0-5460 (5461 slots) master 1 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[ERR] Not all 16384 slots are covered by nodes.[root@server2 ~]# redis-server /usr/local/redis-cluster/30005/redis.conf ##启动30005[root@server2 ~]# redis-cli -c -p 30006 get name"qwe"因为我们停掉了一对主从（30002，30005），导致16384个槽位不满，集群就无法工作了
所以在实际应用中，要是有三台服务器，每台两个节点，则互相交叉配置主从，这样即使一台服务器挂了，集群仍可工作，切忌同一台服务器上是对应的主从节点
3.槽位分配
[root@server2 ~]# redis-server /usr/local/redis-cluster/30002/redis.conf [root@server2 ~]# redis-server /usr/local/redis-cluster/30007/redis.conf [root@server2 ~]# redis-server /usr/local/redis-cluster/30008/redis.conf [root@server2 ~]# redis-trib.rb add-node 127.0.0.1:30007 127.0.0.1:30001 ##将30007加入集群[root@server2 ~]# redis-trib.rb check 127.0.0.1:30006>>> Performing Cluster Check (using node 127.0.0.1:30006)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6dM: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots:5461-10922 (5462 slots) master 1 additional replica(s)M: e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30007 slots: (0 slots) master ##新加入的30007为master，但是集群为奇数个，它没有slave，并且没有分配槽位 0 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10923-16383 (5461 slots) master 1 additional replica(s)M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:0-5460 (5461 slots) master 1 additional replica(s)S: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots: (0 slots) slave replicates e9e0cd3990840c29ae279ddc20b3f946b72a383b[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@server2 ~]# redis-trib.rb reshard --from all --slots 100 --to e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30001 ##给300007分配100个槽位，从master中平均分配[root@server2 ~]# redis-trib.rb check 127.0.0.1:30006>>> Performing Cluster Check (using node 127.0.0.1:30006)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6dM: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots:5495-10922 (5428 slots) master 1 additional replica(s)M: e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30007 slots:0-32,5461-5494,10923-10955 (100 slots) master 0 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:10956-16383 (5428 slots) master 1 additional replica(s)M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:33-5460 (5428 slots) master 1 additional replica(s)S: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots: (0 slots) slave replicates e9e0cd3990840c29ae279ddc20b3f946b72a383b[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@server2 ~]# redis-trib.rb rebalance --threshold 1 127.0.0.1:30001 ##平均分配槽位，如果新加入集群的节点没有像上面那样手动分配槽位（即槽位为空）的时候，要加上--use-empty-masters 参数[root@server2 ~]# redis-trib.rb check 127.0.0.1:30006>>> Performing Cluster Check (using node 127.0.0.1:30006)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6dM: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots:6827-10922 (4096 slots) master 1 additional replica(s)M: e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30007 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 0 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:12288-16383 (4096 slots) master 1 additional replica(s)M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots: (0 slots) slave replicates e9e0cd3990840c29ae279ddc20b3f946b72a383b[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.[root@server2 ~]# redis-trib.rb add-node --slave --master-id e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30008 127.0.0.1:30001 ##添加30008作为30007的从[root@server2 ~]# redis-trib.rb check 127.0.0.1:30006>>> Performing Cluster Check (using node 127.0.0.1:30006)S: e185c034df6eb0b5e513d15e30777463ceef82a8 127.0.0.1:30006 slots: (0 slots) slave replicates 42d80140057ce31ec972b7079e9cb20d7f0f5ca6S: 7d285e51748c1683599dd31c6e27caf806867e2f 127.0.0.1:30001 slots: (0 slots) slave replicates 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6dS: 52a505926e07fb62bf6019627c64ef334734b587 127.0.0.1:30008 slots: (0 slots) slave replicates e3529eb3ae9bc3b5c4f8d1b429c19751da371d2bM: e9e0cd3990840c29ae279ddc20b3f946b72a383b 127.0.0.1:30005 slots:6827-10922 (4096 slots) master 1 additional replica(s)M: e3529eb3ae9bc3b5c4f8d1b429c19751da371d2b 127.0.0.1:30007 slots:0-1364,5461-6826,10923-12287 (4096 slots) master 1 additional replica(s)M: 42d80140057ce31ec972b7079e9cb20d7f0f5ca6 127.0.0.1:30003 slots:12288-16383 (4096 slots) master 1 additional replica(s)M: 2382b55e7e8f2d4119ed265e1aa4e1d10e9c6a6d 127.0.0.1:30004 slots:1365-5460 (4096 slots) master 1 additional replica(s)S: 8504c51e42a572219cc95fb3510d8156ed274a69 127.0.0.1:30002 slots: (0 slots) slave replicates e9e0cd3990840c29ae279ddc20b3f946b72a383b[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.四、Redis 作为 mysql 的缓存服务器
1.角色分配
2.安装lnmp环境
[root@server2 ~]# killall redis-server ##停掉刚才的集群[root@server2 ~]# yum install -y nginx php php-fpm php-cli php-common php-gd php-mbstring php-mysql php-pdo php-devel mysql mysql-server3.安装 php 的 redis 扩展
下载地址：https://github.com/owlient/phpredis
[root@server2 ~]# unzip phpredis-master.zip[root@server2 ~]# cd phpredis-master[root@server2 phpredis-master]# phpize Configuring for:PHP Api Version: 20090626Zend Module Api No: 20090626Zend Extension Api No: 220090626[root@server2 phpredis-master]# ./configure[root@server2 phpredis-master]# make && make install[root@server2 ~]# vim /etc/nginx/conf.d/default.confserver { listen 80; server_name localhost; root /usr/share/nginx/html; include /etc/nginx/default.d/*.conf; location / { root html; index index.php index.html index.htm; } location ~ \.php$ { root html; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME /usr/share/nginx/html$fastcgi_script_name; include fastcgi_params; }}[root@server2 ~]# vim /etc/php.ini; extension=msql.soextension=redis.so[root@server2 ~]# /etc/init.d/php-fpm start[root@server2 ~]# service nginx start[root@server2 ~]# echo "<?php phpinfo() ?>" > /usr/share/nginx/html/index.php网页访问：http://172.25.27.2
要看到加载redis模块
[root@server2 ~]# vim /usr/share/nginx/html/test.php<?php $redis = new Redis(); $redis->connect('172.25.27.3',6379) or die ("could net connect redis server"); # $query = "select * from test limit 9"; $query = "select * from test"; for ($key = 1; $key < 10; $key++) { if (!$redis->get($key)) { $connect = mysql_connect('173.25.27.4','redis','redhat'); mysql_select_db(test); $result = mysql_query($query); //如果没有找到$key,就将该查询sql的结果缓存到redis while ($row = mysql_fetch_assoc($result)) { $redis->set($row['id'],$row['name']); } $myserver = 'mysql'; break; } else { $myserver = "redis"; $data[$key] = $redis->get($key); } } echo $myserver; echo "<br>"; for ($key = 1; $key < 10; $key++) { echo "number is <b><font color=#FF0000>$key</font></b>"; echo "<br>"; echo "name is <b><font color=#FF0000>$data[$key]</font></b>"; echo "<br>"; }?>4.配置 mysql
[root@server4 ~]# yum install -y mysql-server[root@server4 ~]# /etc/init.d/mysqld start[root@server4 ~]# vim test.sqluse test;CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9');#DELIMITER $$#CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN# SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`));# END$$#DELIMITER ;[root@server4 ~]# mysql<test.sql[root@server4 ~]# mysqlmysql> select * from test.test;+----+-------+| id | name |+----+-------+| 1 | test1 || 2 | test2 || 3 | test3 || 4 | test4 || 5 | test5 || 6 | test6 || 7 | test7 || 8 | test8 || 9 | test9 |+----+-------+9 rows in set (0.00 sec)mysql> grant all on test.* to redis@"%" identified by "redhat";5.安装redis
再server3上安装，安装好修改bind之后直接启动即可
[root@server3 redis-4.0.2]# make && make install[root@server3 redis-4.0.2]# ./utils/install_server.sh[root@server3 redis-4.0.2]# vim /etc/redis/6379.confbind 0.0.0.0[root@server3 redis-4.0.2]# /etc/init.d/redis_6379 restart网页访问 http://172.25.27.2/test.php
到这里,我们已经实现了 redis 作为 mysql 的缓存服务器,但是如果更新了 mysql , redis中仍然会有对应的 KEY,数据就不会更新,此时就会出现 mysql 和 redis 数据不一致的情况。所以接下来就要通过 mysql 触发器将改变的数据同步到 redis 中。
五、配置 gearman 实现数据同步
1.Gearman简介
Gearman 是一个支持分布式的任务分发框架:
Gearman Job Server: Gearman 核心程序,需要编译安装并以守护进程形式运行在后台。
Gearman Client:可以理解为任务的请求者。
Gearman Worker:任务的真正执行者,一般需要自己编写具体逻辑并通过守护进程方式运行,Gearman Worker 接收到 Gearman Client 传递的任务内容后,会按顺序处理。
大致流程:下面要编写的 mysql 触发器,就相当于 Gearman 的客户端。修改表,插入表就相当于直接下发任务。然后通过 lib_mysqludf_json UDF 库函数将关系数据映射为 JSON 格式,然后在通过 gearman-mysql-udf 插件将任务加入到 Gearman 的任务队列中,最后通过redis_worker.php,也就是 Gearman 的 worker 端来完成 redis 数据库的更新。
2.安装 gearman 软件包
[root@server2 redis]# yum install gearmand-1.1.8-2.el6.x86_64.rpm libgearman-* libevent-*[root@server1 redis]# /etc/init.d/gearmand start[root@server2 redis]# tar -zxf gearman-1.1.2.tgz[root@server2 redis]# cd gearman-1.1.2[root@server2 gearman-1.1.2]# phpize[root@server2 gearman-1.1.2]# ./configure --with-php-config=/usr/bin/php-config[root@server1 gearman-1.1.2]# make && make instal[root@server2 gearman-1.1.2]# cd /etc/php.d/[root@server2 php.d]# sed 's/mysql/gearman/g' mysql.ini > gearman.ini[root@server2 php.d]# /etc/init.d/php-fpm reload[root@server2 php.d]# php -m | grep gearmangearman3.安装 lib_mysqludf_json
[root@server4 ~]# yum install -y mysql-devel gcc[root@server4 ~]# wget https://codeload.github.com/mysqludf/lib_mysqludf_json/zip/master[root@server4 ~]# unzip master[root@server4 ~]# cd lib_mysqludf_json-master/[root@server4 lib_mysqludf_json-master]# gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c[root@server4 lib_mysqludf_json-master]# mysql -e "show global variables like 'plugin_dir'" ##查看 mysql 的模块目录+---------------+-------------------------+| Variable_name | Value |+---------------+-------------------------+| plugin_dir | /usr/lib64/mysql/plugin |+---------------+-------------------------+[root@server4 lib_mysqludf_json-master]# cp lib_mysqludf_json.so /usr/lib64/mysql/plugin/ ##拷贝 lib_mysqludf_json.so 模块[root@server4 lib_mysqludf_json-master]# mysqlmysql> CREATE FUNCTION json_object RETURNS STRING SONAME 'lib_mysqludf_json.so';mysql> select * from mysql.func;+-------------+-----+----------------------+----------+| name | ret | dl | type |+-------------+-----+----------------------+----------+| json_object | 0 | lib_mysqludf_json.so | function |+-------------+-----+----------------------+----------+1 row in set (0.00 sec)4.安装 gearman-mysql-udf
这个插件是用来管理调用 Gearman 的分布式的队列。
https://launchpad.net/gearman-mysql-udf
[root@server4 ~]# wget http://dl.fedoraproject.org/pub/epel/6/x86_64//libgearman-1.1.8-2.el6.x86_64.rpm[root@server4 ~]# wget http://dl.fedoraproject.org/pub/epel/6/x86_64//libgearman-devel-1.1.8-2.el6.x86_64.rpm[root@server4 ~]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/libevent-devel-1.4.13-4.el6.x86_64.rpm[root@server4 ~]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/libevent-doc-1.4.13-4.el6.noarch.rpm[root@server4 ~]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/libevent-headers-1.4.13-4.el6.noarch.rpm[root@server4 ~]# yum install -y libgearman-devel-1.1.8-2.el6.x86_64.rpm libgearman-1.1.8-2.el6.x86_64.rpm libevent-devel-1.4.13-4.el6.x86_64.rpm libevent-headers-1.4.13-4.el6.noarch.rpm[root@server4 ~]# wget https://launchpadlibrarian.net/104246122/gearman-mysql-udf-0.6.tar.gz[root@server4 ~]# tar -zxf gearman-mysql-udf-0.6.tar.gz[root@server4 ~]# cd gearman-mysql-udf-0.6[root@server4 gearman-mysql-udf-0.6]# ./configure --with-mysql --libdir=/usr/lib64/mysql/plugin/[root@server4 gearman-mysql-udf-0.6]# make && make install5.注册 UDF 函数
[root@server4 gearman-mysql-udf-0.6]# mysqlmysql> CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so';Query OK, 0 rows affected (0.00 sec)mysql> CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so';Query OK, 0 rows affected (0.00 sec)mysql> SELECT gman_servers_set('172.25.27.2:4730'); ##指定 gearman 的服务信息+--------------------------------------+| gman_servers_set('172.25.27.2:4730') |+--------------------------------------+| 172.25.27.2:4730 |+--------------------------------------+1 row in set (0.00 sec)mysql> select * from mysql.func; ##查看函数+--------------------+-----+-------------------------+----------+| name | ret | dl | type |+--------------------+-----+-------------------------+----------+| json_object | 0 | lib_mysqludf_json.so | function || gman_do_background | 0 | libgearman_mysql_udf.so | function || gman_servers_set | 0 | libgearman_mysql_udf.so | function |+--------------------+-----+-------------------------+----------+6.编写 mysql 触发器(根据实际情况编写)
[root@server4 ~]# vim test.sqluse test;#CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;#INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9');DELIMITER $$CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); END$$DELIMITER ;[root@server4 ~]# mysql<test.sql [root@server4 ~]# mysql -e 'SHOW TRIGGERS FROM test\G'*************************** 1. row *************************** Trigger: datatoredis Event: UPDATE Table: test Statement: BEGIN SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); END Timing: AFTER Created: NULL sql_mode: Definer: root@localhostcharacter_set_client: latin1collation_connection: latin1_swedish_ci Database Collation: latin1_swedish_ci7.编写 gearman 的 worker 端
[root@server2 ~]# vim /usr/local/worker.php<?php$worker = new GearmanWorker();$worker->addServer();$worker->addFunction('syncToRedis', 'syncToRedis');$redis = new Redis();$redis->connect('172.25.27.3', 6379);while($worker->work());function syncToRedis($job){ global $redis; $workString = $job->workload(); $work = json_decode($workString); if(!isset($work->id)){ return false; } $redis->set($work->id, $work->name); #这条语句就是将 id 作 KEY 和name 作 VALUE 分开存储,需要和前面写的 php 测试代码的存取一致。}?>[root@server2 ~]# nohup php /usr/local/worker.php & ##后台运行 worker8.测试
更新 mysql 中的数据
[root@server4 ~]# mysql -e "update test.test set name='hello' where id=1"查看 redis
[root@server3 ~]# redis-cli get 1"hello"刷新测试页面数据同步
阅读全文
1 0
- Redis 集群及将 Redis 作为 mysql 的缓存服务器实战
- Redis与mysql的不同:如何将Redis作为Mysql的缓存服务器
- Redis作为MySQL缓存服务器的使用
- redis作为缓存服务器
- redis作为mysql的缓存服务器(读写分离)
- redis作为mysql的缓存服务器(读写分离)
- Windows下Redis作为Mysql的缓存服务器-读写分离
- redis作为mysql的缓存服务器(读写分离)
- Redis 作为缓存服务器的配置
- Redis 作为缓存服务器的配置
- Redis 作为缓存服务器的配置
- Redis 作为缓存服务器的配置
- Redis 作为缓存服务器的配置
- Redis 作为缓存服务器的配置
- 用Redis作为Mysql数据库的缓存(-)
- redis cluster && Redis 作 mysql 的缓存服务器
- 用Redis作为Mysql数据库的缓存(二）
- 初学Redis（2）——用Redis作为Mysql数据库的缓存
- MySQL 5.7安装步骤
- Spring框架中的设计模式(三)
- java 删除目录（文件）以及子目录（文件）代码，请参考，谢谢！
- 条款23：宁以non-member、non-friend替换member函数
- java小程序——简单井字游戏双人版
- Redis 集群及将 Redis 作为 mysql 的缓存服务器实战
- hadoop学习笔记003（感受上传到dfs和hadoop统计计算）
- java多态，抽象类和接口
- 训练日记-31
- 数组和指针十个问答
- 线性判别分析（LDA）, 主成分分析(PCA)
- Uva247（tarjan模板题）
- 第2节:如何成为scala高手
- 按位操作符与背包问题的遍历
