redis key的过期机制

本文为阅读 http://www.redis.io/commands/expire 做的一些记录总结

redis是一个 高性能的key-value数据库.我们可以对其中的key设置一个过期时间,相关命令如下:

1.在创建一个key value 的时候同时设置过期时间

比如: set key1 value1 EX 60 表示: 我们想要将这个 key value 放到redis中,并在其中保存60秒.

set key2 value2 PX 60 表示:我们想要将这个key value放到redis中,并在其中保存 60毫秒

2.对于已经存在的key value, 

比如: 事先有 set key3 value3

我们想为 key3 设置一个过期时间, 我们可以采用以下命令:  

expire  key3 60  表示: 我们要对已经存在的kye3 设置为 在 60秒之后(无效,或者被删除)

我们可以查看一个key当前的 存活(有效?)时间.

使用ttl key4 即可查看剩余有效时间.

==============================以上命令,没有说明返回值表示的意思================

首先说到 "过期", 比如说一个面包, 其保质期为1天. 那么, 我们怎么判定它的有效期?一般情况, 会在做好面包的时候,在标签上打上一个生产日期, 在我们在消费这个面包的时候, 会把我们当前的日期, 和面包的标签上的有效期做一个对比,假如时间差超过1天, 那么我们就判定这个面包过期了, 就是说,我们为面包设置一个基准点.其实, 也可能有另外一种实现的方式,就是在每个面包做好之后,在上面加一个定期器, 我们在消费这个面包的时候,只要看这个定期器中的显示,就可以知道这个面包是否已经过期.

======================

有了上面的一段啰嗦,现在我们就能对redis中的过期机制,起码有两种实现方式, 方式1, 保存一个起始时间,就是面包例子中的打上生产日期, 附上有效时间. 方式2.每一个key内维护一个类似定时器的东西...(这个做起来,维护成本就太大了...,与高性能的旗号违背), redis中key的过期信息,就是通过保存一个过期时间和起始时间信息来维护的.(说明:redis 2.6及其以上版本用毫秒为单位保存 起始时间,2.6以前版本用 unix时间戳保存,所以存在的误差也可能不一样, 2.4版本误差在1秒以内,2.6版本误差在1毫秒以内)

这就有一个有趣的事情, 就是操作的时候,务必确保当前跑redis的机子上的时间是正确的,(一时间联想到了linux下改个系统时间都需要输入管理员密码的用意何在,防止程序随意修改系统时间,可能会导致一部分需要依赖系统时间的程序无法运行)

做了一个小实验: 设置一个key的有效期为60秒,在60秒内将计算机的时间改到1分钟之后,那么这个key 马上失效(突然联想到之前有些应用时间过了就用不了了....so... ) 

所以, 在我们每次用get 命令去取一个key的时候,先把key找到,再判断这个key是否已经过期(自动联想我们看一个面包是否已经过期...).如果已经过期了, redis就不给我们返回对应的value值了. 这是redis 处理key过期的active方式, 但是 仅仅依靠这种方式是不够的,可以想象,假如,我们放置很多具有有效期的key value在redis中,仅仅是每次都判断一个key是否已经过期,那么到最后,系统中就会存在许多垃圾(已经过期,但是又没有人去清理...). 所以,redis,还会每秒钟执行10次以下的流程:

1.随机抽取100个设定了有效期的key,检查其有效期,如果已经过期,则将其删除.

2.如果抽取到的100个key中超过25个已经过期,那么返回步骤1,

这就是redis中清除已经过期的key的 passive 方法.

=====================================

可能会影响(或者使用者觉得会影响)key有效期的操作.

1.会删除有效期的操作.

 del 命令.

expire命令

set 命令

getset 命令

另外,其他改变值的操作都是不会修改有效期的, (文章原文:. This means that all the operations that conceptually alter the value stored at the key without replacing it with a new one will leave the timeout untouched. For instance, incrementing the value of a key with INCR, pushing a new value into a list with LPUSH, or altering the field value of a hash with HSET are all operations that will leave the timeout untouched.)

此外,还有一个命令,rename . 假如存在 key1 value1 .  并且key1存在一个有效期, 那么使用rename key1 key2.这时候,key2的生存时间也会和原key1保持一致. 另外一种情况: 存在 key1, value1, key2,value2.  并且key1存在一个有效期,  那么使用rename key1 key2.这时候 key2对应的value是 value1, value2将被覆盖,而且 key2的所有characteristics将会被 原key1的所有characteristics. (文章原文: it does not matter if the original Key_A had a timeout associated or not, the new key Key_Awill inherit all the characteristics of Key_B)

这一小段一开始让我疑惑:

If a key is renamed with RENAME, the associated time to live is transferred to the new key name.

If a key is overwritten by RENAME, like in the case of an existing key Key_A that is overwritten by a call likeRENAME Key_B Key_A, it does not matter if the original Key_A had a timeout associated or not, the new key Key_Awill inherit all the characteristics of Key_B.

有   ranamed with rename

...overwritten by rename ... 

===================================

redis删除已过期节点时对于  aof文件以及 主从节点集群时的处理方式

当一个key过期时,redis会同步删除对应的aof文件,假如key分布在主从集群的节点时, 也会同步删除 对应的 slaves上的key.值得注意的是, slaves不会主动删除已经过期的key,而会一直等待master的删除命令. 只有当一个slave成为了master之后,才会主动删除已经过期的key.

另外,文章也给了一个应用的例子,但是英语水平以及开发经验有限,..无法理解,原文如下:

Pattern: Navigation session

Imagine you have a web service and you are interested in the latest N pages recently visited by your users, such that each adjacent page view was not performed more than 60 seconds after the previous. Conceptually you may think at this set of page views as a Navigation session if your user, that may contain interesting information about what kind of products he or she is looking for currently, so that you can recommend related products.

You can easily model this pattern in Redis using the following strategy: every time the user does a page view you call the following commands:

MULTIRPUSH pagewviews.user:<userid> http://.....EXPIRE pagewviews.user:<userid> 60EXEC

If the user will be idle more than 60 seconds, the key will be deleted and only subsequent page views that have less than 60 seconds of difference will be recorded.

This pattern is easily modified to use counters using INCR instead of lists using RPUSH.

...

=============================================================================

鉴于个人打开原页面的时候,有时候速度非常慢,特将原文附上:

EXPIRE key seconds

Related commands

• DEL

• DUMP

• EXISTS

• EXPIRE

• EXPIREAT

• KEYS

• MIGRATE

• MOVE

• OBJECT

• PERSIST

• PEXPIRE

• PEXPIREAT

• PTTL

• RANDOMKEY

• RENAME

• RENAMENX

• RESTORE

• SCAN

• SORT

• TTL

• TYPE

Available since 1.0.0.

Time complexity: O(1)

Set a timeout on key. After the timeout has expired, the key will automatically be deleted. A key with an associated timeout is often said to be volatile in Redis terminology.

The timeout is cleared only when the key is removed using the DEL command or overwritten using the SET orGETSET commands. This means that all the operations that conceptually alter the value stored at the key without replacing it with a new one will leave the timeout untouched. For instance, incrementing the value of a key with INCR, pushing a new value into a list with LPUSH, or altering the field value of a hash with HSET are all operations that will leave the timeout untouched.

The timeout can also be cleared, turning the key back into a persistent key, using the PERSIST command.

If a key is renamed with RENAME, the associated time to live is transferred to the new key name.

If a key is overwritten by RENAME, like in the case of an existing key Key_A that is overwritten by a call likeRENAME Key_B Key_A, it does not matter if the original Key_A had a timeout associated or not, the new key Key_Awill inherit all the characteristics of Key_B.

Refreshing expires

It is possible to call EXPIRE using as argument a key that already has an existing expire set. In this case the time to live of a key is updated to the new value. There are many useful applications for this, an example is documented in the Navigation session pattern section below.

Differences in Redis prior 2.1.3

In Redis versions prior 2.1.3 altering a key with an expire set using a command altering its value had the effect of removing the key entirely. This semantics was needed because of limitations in the replication layer that are now fixed.

Return value

Integer reply, specifically:

• 1 if the timeout was set.

• 0 if key does not exist or the timeout could not be set.

Examples

redis> SET mykey "Hello"

OK

redis> EXPIRE mykey 10

(integer) 1

redis> TTL mykey

(integer) 10

redis> SET mykey "Hello World"

OK

redis> TTL mykey

(integer) -1

redis> 

Pattern: Navigation session

Imagine you have a web service and you are interested in the latest N pages recently visited by your users, such that each adjacent page view was not performed more than 60 seconds after the previous. Conceptually you may think at this set of page views as a Navigation session if your user, that may contain interesting information about what kind of products he or she is looking for currently, so that you can recommend related products.

You can easily model this pattern in Redis using the following strategy: every time the user does a page view you call the following commands:

MULTIRPUSH pagewviews.user:<userid> http://.....EXPIRE pagewviews.user:<userid> 60EXEC

If the user will be idle more than 60 seconds, the key will be deleted and only subsequent page views that have less than 60 seconds of difference will be recorded.

This pattern is easily modified to use counters using INCR instead of lists using RPUSH.

Appendix: Redis expires

Keys with an expire

Normally Redis keys are created without an associated time to live. The key will simply live forever, unless it is removed by the user in an explicit way, for instance using the DEL command.

The EXPIRE family of commands is able to associate an expire to a given key, at the cost of some additional memory used by the key. When a key has an expire set, Redis will make sure to remove the key when the specified amount of time elapsed.

The key time to live can be updated or entirely removed using the EXPIRE and PERSIST command (or other strictly related commands).

Expire accuracy

In Redis 2.4 the expire might not be pin-point accurate, and it could be between zero to one seconds out.

Since Redis 2.6 the expire error is from 0 to 1 milliseconds.

Expires and persistence

Keys expiring information is stored as absolute Unix timestamps (in milliseconds in case of Redis version 2.6 or greater). This means that the time is flowing even when the Redis instance is not active.

For expires to work well, the computer time must be taken stable. If you move an RDB file from two computers with a big desync in their clocks, funny things may happen (like all the keys loaded to be expired at loading time).

Even running instances will always check the computer clock, so for instance if you set a key with a time to live of 1000 seconds, and then set your computer time 2000 seconds in the future, the key will be expired immediately, instead of lasting for 1000 seconds.

How Redis expires keys

Redis keys are expired in two ways: a passive way, and an active way.

A key is actively expired simply when some client tries to access it, and the key is found to be timed out.

Of course this is not enough as there are expired keys that will never be accessed again. This keys should be expired anyway, so periodically Redis test a few keys at random among keys with an expire set. All the keys that are already expired are deleted from the keyspace.

Specifically this is what Redis does 10 times per second:

1. Test 100 random keys from the set of keys with an associated expire.

2. Delete all the keys found expired.

3. If more than 25 keys were expired, start again from step 1.

This is a trivial probabilistic algorithm, basically the assumption is that our sample is representative of the whole key space, and we continue to expire until the percentage of keys that are likely to be expired is under 25%

This means that at any given moment the maximum amount of keys already expired that are using memory is at max equal to max amount of write operations per second divided by 4.

How expires are handled in the replication link and AOF file

In order to obtain a correct behavior without sacrificing consistency, when a key expires, a DEL operation is synthesized in both the AOF file and gains all the attached slaves. This way the expiration process is centralized in the master instance, and there is no chance of consistency errors.

However while the slaves connected to a master will not expire keys independently (but will wait for the DEL coming from the master), they'll still take the full state of the expires existing in the dataset, so when a slave is elected to a master it will be able to expire the keys independently, fully acting as a master.

====================================================

总结:虽然设置一个redis的key设置过期对于整个redis系统来说,不是一个大的功能,但是要将一个小功能做好,也是需要充分考虑到方方面面.另外,抛一块转头: 在web系统中,session一般也是有一个过期时间的,那么在session的过期时间又是如何实现的呢?====  java ee...