Reids Cluster 部署

目前，项目用的redis主备，感觉超不爽，很多局限性，特别是在 redis master 宕机之后，维护非常麻烦，寻思着弄一个redis集群，Redis3.0+后终于提供了cluster方案。

架构：Redis Cluster采用无中心结构，每个节点都保存数据和整个集群的状态

每个节点都和其他所有节点连接，这些连接保持活跃

使用gossip协议传播信息以及发现新节点

node不作为client请求的代理，client根据node返回的错误信息重定向请求

数据分布：预分桶

预分好16384个桶，根据 CRC16(key) mod 16384的值，决定将一个key放到哪个桶中

每个Redis物理结点负责一部分桶的管理，当发生Redis节点的增减时，调整桶的分布即可

例如，假设Redis Cluster三个节点A/B/C，则

Node A 包含桶的编号可以为: 0 到 5500.

Node B 包含桶的编号可以为: 5500 到 11000.

Node C包含桶的编号可以为: 11001 到 16384.

当发生Redis节点的增减时，调整桶的分布即可。

预分桶的方案介于“硬Hash”和“一致性Hash”之间，牺牲了一定的灵活性，但相比“一致性Hash“，数据的管理成本大大降低

安装配置

安装编译环境

$yum install gcc gcc-c++ gcc-g77 flex bison autoconf automake bzip2-devel zlib-devel ncurses-devel libjpeg-devel libpng-devel libtiff-devel freetype-devel pam-devel openssl-devel libxml2-devel gettext-devel pcre-devel tcl -y

安装高版本ruby (yum安装的版本过低)

卸载老版本:

$yum erase ruby ruby-libs ruby-mode ruby-rdoc ruby-irb ruby-ri ruby-docs  $wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.1.tar.gz  $tar -zxvf ...  $./configure  $make && make install$gem sources -r https://rubygems.org/$gem sources -a https://ruby.taobao.org/$gem sources –l

安装rubygems

$wget http://production.cf.rubygems.org/rubygems/rubygems-2.2.2.tgz$ruby setup.rb

安装redis

$wget http://download.redis.io/releases/redis-3.2.0.tar.gz$make MALLOC=libc$cd src$cp redis-server /usr/sbin/$cp redis-cli /usr/sbin/$cp redis-trib.rb /usr/sbin/

utils目录下有命令脚本的样例参考。

配置

• 创建用户和目录:

$adduser -M -U redis$mkdir /var/lib/redis$mkdir /etc/redis$chown redis:redis /var/lib/redis$chown redis:redis /etc/redis

• 修改系统配置:

$echo 1 > /proc/sys/vm/overcommit_memory$vi /etc/sysctl.conf添加vm.overcommit_memory=1$echo 511 > /proc/sys/net/core/somaxconn$echo never > /sys/kernel/mm/transparent_hugepage/enabled$vi /etc/rc.local添加echo never > /sys/kernel/mm/transparent_hugepage/enabled编辑redis配置文件7000~7005$vi /etc/redis/7000.conf

7000.conf(其它配置一致，port7000~7005)

$bind 127.0.0.1$port 7000$pidfile /var/run/redis/7000.pid$loglevel notice$logfile /var/log/redis/7000.log$daemonize yes$dir /var/lib/redis/7000$cluster-enabled yes$cluster-config-file nodes.conf$cluster-node-timeout 5000$appendonly yes$appendfsync everysec

$repl-disable-tcp-nodelay yes

启动文件/etc/init.d/redis-cluster

>

#!/bin/bash# Source function library.. /etc/rc.d/init.d/functions# SettingsPORT=7000TIMEOUT=2000NODES=6REPLICAS=1# You may want to put the above config parameters into config.sh in order to# override the defaults without modifying this script.if [ -a config.sh ]then   source "config.sh"fi# Computed varsENDPORT=$((PORT+NODES))if [ "$1" == "start" ]then   while [ $((PORT < ENDPORT)) != "0" ]; do       #PORT=$((PORT+1))       echo "Starting $PORT"       daemon --user redis "/usr/sbin/redis-server /etc/redis/${PORT}.conf"       PORT=$((PORT+1))   done   exit 0fiif [ "$1" == "create" ]then   HOSTS=""   while [ $((PORT < ENDPORT)) != "0" ]; do       #PORT=$((PORT+1))       HOSTS="$HOSTS 127.0.0.1:$PORT"       PORT=$((PORT+1))   done   /usr/sbin/redis-trib.rb create --replicas $REPLICAS $HOSTS   exit 0fiif [ "$1" == "stop" ]then   while [ $((PORT < ENDPORT)) != "0" ]; do       #PORT=$((PORT+1))       echo "Stopping $PORT"       /usr/sbin/redis-cli -p $PORT shutdown nosave       PORT=$((PORT+1))   done   exit 0fiif [ "$1" == "watch" ]then   while [ 1 ]; do       clear       date       PORTTMP=$PORT       while [ $((PORTTMP < ENDPORT)) != "0" ]; do           /usr/sbin/redis-cli -p $PORTTMP cluster nodes | head -30           PORTTMP=$((PORTTMP+1))       done       sleep 2   done   exit 0fiif [ "$1" == "tail" ]then   INSTANCE=$2   PORT=$((PORT+INSTANCE))   tail -f /var/log/redis/${PORT}.log   exit 0fiif [ "$1" == "call" ]then   while [ $((PORT < ENDPORT)) != "0" ]; do       #PORT=$((PORT+1))       /usr/sbin/redis-cli -p $PORT $2 $3 $4 $5 $6 $7 $8 $9       PORT=$((PORT+1))   done   exit 0fiif [ "$1" == "clean" ]then   while [ $((PORT < ENDPORT)) != "0" ]; do       rm -rf /var/log/redis/${PORT}.log       rm -rf /var/lib/redis/${PORT}/*       PORT=$((PORT+1))   done   exit 0fiecho "Usage: $0 [start|create|stop|watch|tail|clean]"echo "start       -- Launch Redis Cluster instances."echo "create      -- Create a cluster using redis-trib create."echo "stop        -- Stop Redis Cluster instances."echo "watch       -- Show CLUSTER NODES output (first 30 lines) of first node."echo "tail <id>   -- Run tail -f of instance at base port + ID."echo "clean       -- Remove all instances data, logs, configs."

一些简单的命令

1. 检查状态

   $redis-trib.rb check 127.0.0.1:7000

2. 添加master节点

   $redis-trib.rb add-node 127.0.0.1:7006

   node:新节点没有包含任何数据， 因为它没有包含任何slot。新加入的加点是一个主节点， 当集群需要将某个从节点升级为新的主节点时， 这个新节点不会被选中

   为新节点分配slot

   $redis-trib.rb reshard 127.0.0.1:7006

3. 添加slave节点
   步骤同上

   redis-cli连接上新节点shell，输入命令：cluster replicate 对应master的mode-id

   >cluster replicate 2b9ebcbd627ff0fd7a7bbcc5332fb09e72788835

4. 删除slave节点

   $redis-trib.rb del-node ip:port '<node-id>'$redis-trib.rb del-node 127.0.0.1:7103 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

---

Python3.5

Redis-py-cluster