redis replication问题一解

公司有个redis比较大，同时又是跨IDC同步，但是最近发现一旦连接断了之后，好久都不能完全同步。

查看了一下log。

123456

 # I/O error trying to sync with MASTER: connection lost[12826] 01 Apr 16:54:38.555 * Connecting to MASTER 10.x.x.x:6379[12826] 01 Apr 16:54:38.555 * MASTER  SLAVE sync started[12826] 01 Apr 16:54:38.621 * Non blocking connect for SYNC fired the event.[12826] 01 Apr 16:54:38.692 * Master replied to PING, replication can continue...[12826] 01 Apr 16:54:45.229 * MASTER  SLAVE sync: receiving 390598473 bytes from master

通过info观察也是一直去master上同步，有时候看着马上就完成了，就报以上的错误。 以前使用redis很老的版本2.4一直都没有这样的问题，自从换了2.6以后才出现这样的问题的。

放狗查了下原来是 client-output-buffer-limit 这个参数导致的。默认是：

1

client-output-buffer-limit slave 256mb 64mb 60

但是不是很明白这个参数的含义，看了redis文档还正有说这个的。

大体意思就是如下：

hard limit是一旦redis到达这个值后会马上关闭client连接。
soft limit是一种依赖于时间的。 比如一个soft limit被设置为32MB 10s, 那就意味着当client的output buffer超过32MB，并且持续10秒钟，那这个连接就会被断开。
默认值就是hard 为256M， soft为 32M 60秒
普通客户端的默认limit为0, 就是任何时候都没有limit，因为普通的client使用阻塞来实现发送命令和接收完整的返回，在发送下一个命令之前，所以在普通的client的情况下关闭连接是不合适的。
但是要特别注意pub/sub客户端，这种方式一次处理和输出的数据都会特别大。
我们可以使用config set 来进行设置，但是注意使用config set的时候不支持MB，GB 这样的单位。

下面这个是redis官网的一个具体说明：
http://redis.io/topics/clients

Output buffers limits
Redis needs to handle a variable-length output buffer for every client, since a command can produce a big amount of data that needs to be transferred to the client.

However it is possible that a client sends more commands producing more output to serve at a faster rate at which Redis can send the existing output to the client. This is especially true with Pub/Sub clients in case a client is not able to process new messages fast enough.

Both the conditions will cause the client output buffer to grow and consume more and more memory. For this reason by default Redis sets limits to the output buffer size for different kind of clients. When the limit is reached the client connection is closed and the event logged in the Redis log file.

There are two kind of limits Redis uses:
•The hard limit is a fixed limit that when reached will make Redis closing the client connection as soon as possible.
•The soft limit instead is a limit that depends on the time, for instance a soft limit of 32 megabytes per 10 seconds means that if the client has an output buffer bigger than 32 megabytes for, continuously, 10 seconds, the connection gets closed.

Different kind of clients have different default limits:
•Normal clients have a default limit of 0, that means, no limit at all, because most normal clients use blocking implementations sending a single command and waiting for the reply to be completely read before sending the next command, so it is always not desirable to close the connection in case of a normal client.
•Pub/Sub clients have a default hard limit of 32 megabytes and a soft limit of 8 megabytes per 60 seconds.
•Slaves have a default hard limit of 256 megabytes and a soft limit of 64 megabyte per 60 second.

It is possible to change the limit at runtime using the CONFIG SET command or in a permanent way using the Redis configuration file redis.conf. See the example redis.conf in the Redis distribution for more information about how to set the limit.