Redis cluster - 乾坤大挪移

Redis cluster 简介

Redis Cluster 101

The ability to automatically split your dataset among multiple nodes. 数据自动完成在不同安装节点的分片。The ability to continue operations when a subset of the nodes are experiencing failures or are unable to communicate with the rest of the cluster.如果分片出现异常，可以保证一定程度的可用性。

Redis Cluster TCP ports（TCP端口）

每个redis集群需要开两个TCP端口，一个为客户端调用端口（可以叫命令端口，比如6379）；另一个为端口为命令端口+10000（可以集群通讯端口，比如16379），这个端口用于集群间节点的通讯（主要完成failure detection, configuration update, failover authorization，其实就是以前哨兵干的活）。

注：命令端口（command port）+10000= 集群通讯端口（cluster bus port） ，而且总是这样。

为确保集群的正常运行，集群中的节点需要做到以下两点：

1.命令端口对需要访问集群的所有客户端可达，同时对集群节点可达。The normal client communication port (usually 6379) used to communicate with clients to be open to all the clients that need to reach the cluster, plus all the other cluster nodes (that use the client port for keys migrations).2.集群通讯端口对集群中所有节点可达。The cluster bus port (the client port + 10000) must be reachable from all the other cluster nodes.The cluster bus uses a different, binary protocol, for node to node data exchange, which is more suited to exchange information between nodes using little bandwidth and processing time.数据通讯和集群通讯采用了不同的协议，集群通讯协议信息交换占用的带宽更小，处理更快。**总结**：读到这里，感觉redis集群功能把原本单独部署的哨兵和redis数据结点功能合到了一起，集群中的每个结点即是哨兵又是数据结点。

Redis Cluster data sharding 数据分片

集群中每个结点被分成了N个hash slots（暂且叫虚拟结点），一个集群中总共有 16384个虚拟结点。数据分发没有采用一致性哈希算法（consistency hashing）。
每个数据结点负责一部分虚拟结点，加入集群中有3个结点：

Node A contains hash slots from 0 to 5500.Node B contains hash slots from 5501 to 11000.Node C contains hash slots from 11001 to 16384.

This allows to add and remove nodes in the cluster easily. For example if I want to add a new node D, I need to move some hash slot from nodes A, B, C to D. Similarly if I want to remove node A from the cluster I can just move the hash slots served by A to B and C. When the node A will be empty I can remove it from the cluster completely.

上面两段懒得翻译了，原理很简单：
可以想象一个服装加工厂（集群），工厂里有１０台服装加工机器（虚拟结点）。工厂里有两名正式员工（Ａ，Ｂ），当然一人负责５台；赶上大订单，业务量大，公司会招一名零时工Ｃ（加机器），新来的零时工会从Ａ，Ｂ那里接过一部分工作；赶上Ａ有事（服务挂了），需要离开，那么Ａ负责的工作，需要分给Ｂ，Ｃ。

Redis Cluster supports multiple key operations as long as all the keys involved into a single command execution (or whole transaction, or Lua script execution) all belong to the same hash slot. The user can force multiple keys to be part of the same hash slot by using a concept called hash tags.

Hash tags are documented in the Redis Cluster specification, but the gist is that if there is a substring between {} brackets in a key, only what is inside the string is hashed, so fo example this{foo}key and another{foo}key are guaranteed to be in the same hash slot, and can be used together in a command with multiple keys as arguments.

redis支持事务，如果保证事务中的所有操作都映射到同一个虚拟结点， 于是引入了hash tags这一个概念来解决这一问题。

Redis Cluster master-slave model 主从模型

通常意义的主从模型，主fail后，从升为主，继续提供服务。

Redis Cluster consistency guarantees一致性

出于性能performances和一致性consistency的综合考虑，redis cluster不能保证强一致性，会出现写失败（ lose writes）的问题。.
场景1：从同步主数据失败，从升为主。
场景2：happens during a network partition where a client is isolated with a minority of instances including at least a master.

Take as an example our 6 nodes cluster composed of A, B, C, A1, B1, C1, with 3 masters and 3 slaves. There is also a client, that we will call Z1.

After a partition occurs, it is possible that in one side of the partition we have A, C, A1, B1, C1, and in the other side we have B and Z1.

Z1 is still able to write to B, that will accept its writes. If the partition heals in a very short time, the cluster will continue normally. However if the partition lasts enough time for B1 to be promoted to master in the majority side of the partition, the writes that Z1 is sending to B will be lost.

Note that there is a maximum window to the amount of writes Z1 will be able to send to B: if enough time has elapsed for the majority side of the partition to elect a slave as master, every master node in the minority side stops accepting writes.

This amount of time is a very important configuration directive of Redis Cluster, and is called the node timeout.

After node timeout has elapsed, a master node is considered to be failing, and can be replaced by one of its replicas. Similarly after node timeout has elapsed without a master node to be able to sense the majority of the other master nodes, it enters an error state and stops accepting writes.

Redis Cluster configuration parameters配置参数说明

cluster-enabled <yes/no>: 是否支持集群.cluster-config-file <filename>: 集群配置文件cluster-node-timeout <milliseconds>: failover时间和集群中节点自检超时时间，如果集群中的每个结点不能和集群中其他大多数节点通讯超过配置时间，主节点将不再接受写入。cluster-slave-validity-factor <factor>: If set to zero, a slave will always try to failover a master, regardless of the amount of time the link between the master and the slave remained disconnected. If the value is positive, a maximum disconnection time is calculated as the node timeout value multiplied by the factor provided with this option, and if the node is a slave, it will not try to start a failover if the master link was disconnected for more than the specified amount of time. For example if the node timeout is set to 5 seconds, and the validity factor is set to 10, a slave disconnected from the master for more than 50 seconds will not try to failover its master. Note that any value different than zero may result in Redis Cluster to be not available after a master failure if there is no slave able to failover it. In that case the cluster will return back available only when the original master rejoins the cluster.cluster-migration-barrier <count>: Minimum number of slaves a master will remain connected with, for another slave to migrate to a master which is no longer covered by any slave. See the appropriate section about replica migration in this tutorial for more information.cluster-require-full-coverage <yes/no>: If this is set to yes, as it is by default, the cluster stops accepting writes if some percentage of the key space is not covered by any node. If the option is set to no, the cluster will still serve queries even if only requests about a subset of keys can be processed.

Creating and using a Redis Cluster

1.按照配置启动redis服务（6个实例）

mkdir cluster-testcd cluster-testmkdir 7000 7001 7002 7003 7004 7005最小配置port 7000cluster-enabled yescluster-config-file nodes.confcluster-node-timeout 5000appendonly yes

注意：that the minimal cluster that works as expected requires to contain at least three master nodes.

2.Creating the cluster创建集群
配置命令redis-trib

./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 \127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

或者如果想尽快体验集群效果，可以执行以下脚本，该脚本会创建一个3主3从的集群：
Redis distribution/utils/create-cluster

create-cluster startcreate-cluster createcreate-cluster stop.

---

注：该武功适用于Redis version 3.0 or higher, 建议先读参考文献【1】，再读【2】。

ref：
1. redis cluster tutorial , http://redis.io/topics/cluster-tutorial
2. redis cluster specification , http://redis.io/topics/cluster-spec