最近在写一个distributed cache，把设计思想分享一下

Source code:  https://github.com/gilbertwang1981/cache.git

Dependency:  https://github.com/gilbertwang1981/commonlib4c.git

Language: C++/JAVA

Cache manager is under development.

• Architecture Overview

  
  

1.        Application could access the cache service through load balance based on the fourth layer of ISO-OSI 7-layer protocol such as LVS etc.

2.        The Cache service provides the API for the application to access to the cached data.

3.        The cached data could be replicated among cache instances.

4.        The cache manager could configure the instance of the cache, collect the information from the cache instance such as CPU, Memory and statistic data etc. and monitor the cache instance.

5.        The cache instance will be run as a process which could be deployed on the same machine.

6.        The cache instance will provide two types of the data storage. The one is that the cached data will be stored on the disk in form of the fastdb data file. The other is that the cached data will be stored in the memory.

• Main Protocol

  
  

1.        The communication protocol based on the TCP long or short connection is binary.

2.        The format of the protocol includes the header and body. The length of the header is 12 bytes. The length of the body is variable. The format of the protocol header is as follows.

Field

Size (byte)

Description

length

4

The total length of the pack

command

4

Command such as

get, set, del and

replication etc.

version

4

Protocol version

 

3.        The format of the body request is as follows.

Field

Size (byte)

Description

Key length

4

The length of the key

Key

Key length

The key

value length

4

The length of the

value(just for set

command)

value

value length

The length of the

value(just for set

command)

timeout

4

The timeout of the

Key (just for set

command)

 

4.        The format of the body response is as follows.

Field

Size (byte)

Description

Result code

4

Result code

description

64

The description for

the cache server

value length

4

The length of the

value(just for set

command)

value

value length

The length of the

value(just for set

command)

 

5.        The format of the replicated message is as follows.

Field

Size (byte)

Description

The length of the

cluster IP address

4

The length

The cluster IP

The length of the

cluster IP address

The cluster of the IP

……

……

……

The body of the

command

variable

get/set/del command

 

• Detail

1.        How was the data expired?

The lazy expiration was applied on the cache. When the application gets the data from the cache, the cache will decide whether the data will be expired.

2.        How the replication will be run?

The command will be forwarded one cache instance for another via a router table in the message. The consistence of the copies will be verified by the cache manager in the cluster.

3.        What is the main data structure of the cache in-memory mode?

M-Tree,the key will be stored on the non-leaf node. The data will be stored on the leaf node.

For example, the key is “abc”, the value is “hello world”and the key is “ab”, the value is “hello world”. The memory layout for this KV pair is as follows.

4.        What is the cache manager?

The cache manager is responsible for:

a)        Managing the cache instances for the cluster.

b)        Monitoring the cache instances for the cluster.

c)        Configuring the cache instance into the cluster.

d)        Removing/Adding the cache instance out of/in service dynamically.

e)        Providing the CLI (Telnet)interface for the remote operator.

f)     Ensuring the data consistence for the cache cluster by the global bitmap.

5.        How the disk-mode implements?

Use fastdb.