基于docker的codis3部署

一、codis3介绍

最新的介绍请参考codis官方github：https://github.com/CodisLabs/codis

二、利用docker搭建codis

codis的各个组件的部署结构如下：

组件名称容器内ip映射宿主机ipzookeeper21812181dashboard1808028080proxy管理端口：11080
服务端口：19000管理端口：21080
服务端口：29000codis-server（四个容器）s1：6379
s2：6379
s3：6379
s4：6379s1：26379
s2：26380
s3：26381
s4：26382fe80808080

1、准备Dockerfile

新建目录：mkdir -p docker/codis

进入codis：cd docker/codis

新建Dockerfile：vi Dockerfile

FROM golang
RUN apt-get update
RUN apt-get install -y vim bash golang
RUN apt-get clean
RUN rm -rf /var/lib/apt/lists/*

ENV GOPATH /gopath
ENV CODISLABS ${GOPATH}/src/github.com/CodisLabs #存放Codis源码目录
ENV CODIS ${CODISLABS}/codis
ENV PATH ${GOPATH}/bin:${PATH}:${CODIS}/bin

WORKDIR ${CODISLABS}
RUN git clone https://github.com/CodisLabs/codis.git #获取codis源码

WORKDIR ${CODIS}
RUN git checkout release3.2 #基于codis3.2release版本构建

WORKDIR ${CODIS}
RUN makeWORKDIR /codis

参考github的例子：https://github.com/CodisLabs/codis/blob/release3.2/Dockerfile

2、在Dockerfile所在目录下构建image

docker build -t codis-image .

构建的过程会非常长，慢慢等吧...

3、构建docker的启动脚本

参考：https://github.com/CodisLabs/codis/blob/release3.2/scripts/docker.sh，但是需要进行如下的修改：

3.1、修改hostip

官方的例子中通过下面的代码获取宿主机的ip地址：

hostip=`ifconfig eth0 | grep "inet " | awk -F " " '{print $2}'`

但是有的宿主机没有eth0网卡，可以是别的名字，所以需要先通过ifconfig看看本机的网卡名称，然后手动执行上面的命令，看能不能正确返回ip地址，如果还有问题，建议最简单的方式直接改为hostip='宿主机ip'

3.2、如果默认是root用户，需要修改docker run 命令，增加--privilieged选项，不然容器无法使用共享卷中的配置文件，会报”权限错误“

修改后的启动脚本如下：

#!/bin/bash
#获取宿主机ip地址
hostip=`ifconfig eth0 | grep "inet " | awk -F " " '{print $2}'`

if [ "x$hostip" == "x" ]; then
    echo "cann't resolve host ip address"
    exit 1
fi

mkdir -p log

case "$1" in
zookeeper)  #启动zookeeper
    docker rm -f "Codis-Z2181" &> /dev/null
    docker run --name "Codis-Z2181" -d \
            --read-only \
            -p 2181:2181 \
            jplock/zookeeper
    ;;

dashboard)  #启动dashboard
    docker rm -f "Codis-D28080" &> /dev/null
    docker run --name "Codis-D28080" -d --privileged \
        --read-only -v `realpath config/dashboard.toml`:/codis/dashboard.toml \
                    -v `realpath log`:/codis/log \
                    -v `realpath log`:/codis/log \
        -p 28080:18080 \
        codis-image \
        codis-dashboard -l log/dashboard.log -c dashboard.toml --host-admin ${hostip}:28080
    ;;

proxy)  #启动proxy
    docker rm -f "Codis-P29000" &> /dev/null
    docker run --name "Codis-P29000" -d --privileged \
        --read-only -v `realpath config/proxy.toml`:/codis/proxy.toml \
                    -v `realpath log`:/codis/log \
        -p 29000:19000 -p 21080:11080 \
        codis-image \
        codis-proxy -l log/proxy.log -c proxy.toml --host-admin ${hostip}:21080 --host-proxy ${hostip}:29000
    ;;

server)  #启动4个redis实例
    for ((i=0;i<4;i++)); do
        let port="26379 + i"
        docker rm -f "Codis-S${port}" &> /dev/null
        docker run --name "Codis-S${port}" -d --privileged \
            -v `realpath config/redis.conf`:/codis/redis.conf \
            -v `realpath log`:/codis/log \
            -p $port:6379 \
            codis-image \
            codis-server redis.conf --logfile log/${port}.log
    done
    ;;

fe)  #启动监控页面
    docker rm -f "Codis-F8080" &> /dev/null
    docker run --name "Codis-F8080" -d --privileged \
         -v `realpath log`:/codis/log \
         -p 8080:8080 \
     codis-image \
     codis-fe -l log/fe.log --zookeeper ${hostip}:2181 --listen=0.0.0.0:8080 --assets=/gopath/src/github.com/CodisLabs/codis/bin/assets
    ;;

cleanup)
    docker rm -f "Codis-D28080" &> /dev/null
    docker rm -f "Codis-P29000" &> /dev/null
    for ((i=0;i<4;i++)); do
        let port="26379 + i"
        docker rm -f "Codis-S${port}" &> /dev/null
    done
    docker rm -f "Codis-Z2181" &> /dev/null
    ;;

*)
    echo "wrong argument(s)"
    ;;

esac

4、准备配置文件

建立config目录，把dashboard.toml、proxy.toml、redis.conf复制到此目录下，配置文件可以从github中获取：https://github.com/CodisLabs/codis/tree/release3.2/config

4.1、dashboard.toml

需要修改coordinator_addr为宿主机的zookeeper的ip与端口

需要修改produce_name

##################################################
# #
# Codis-Dashboard #
# #
##################################################

# Set Coordinator, only accept "zookeeper" & "etcd" & "filesystem".
# Quick Start
coordinator_name = "zookeeper"
coordinator_addr = "192.168.28.68:2181"

# Set Codis Product Name/Auth.
product_name = "codis-demo"
product_auth = ""

# Set bind address for admin(rpc), tcp only.
admin_addr = "0.0.0.0:18080"

# Set arguments for data migration (only accept 'sync' & 'semi-async').
migration_method = "semi-async"
migration_parallel_slots = 100
migration_async_maxbulks = 200
migration_async_maxbytes = "32mb"
migration_async_numkeys = 500
migration_timeout = "30s"

# Set configs for redis sentinel.
sentinel_quorum = 2
sentinel_parallel_syncs = 1
sentinel_down_after = "30s"
sentinel_failover_timeout = "5m"
sentinel_notification_script = ""
sentinel_client_reconfig_script = ""

4.2、proxy.toml

需要修改product_name，与dashboard.toml中的一致

##################################################
#                                                #
#                  Codis-Proxy                   #
#                                                #
##################################################

# Set Codis Product Name/Auth.
product_name = "codis-demo"
product_auth = ""

# Set auth for client session
#   1. product_auth is used for auth validation among codis-dashboard,
#      codis-proxy and codis-server.
#   2. session_auth is different from product_auth, it requires clients
#      to issue AUTH <PASSWORD> before processing any other commands.
session_auth = ""

# Set bind address for admin(rpc), tcp only.
admin_addr = "0.0.0.0:11080"

# Set bind address for proxy, proto_type can be "tcp", "tcp4", "tcp6", "unix" or "unixpacket".
proto_type = "tcp4"
proxy_addr = "0.0.0.0:19000"

# Set jodis address & session timeout
#   1. jodis_name is short for jodis_coordinator_name, only accept "zookeeper" & "etcd".
#   2. jodis_addr is short for jodis_coordinator_addr
#   3. proxy will be registered as node:
#        if jodis_compatible = true (not suggested):
#          /zk/codis/db_{PRODUCT_NAME}/proxy-{HASHID} (compatible with Codis2.0)
#        or else
#          /jodis/{PRODUCT_NAME}/proxy-{HASHID}
jodis_name = ""
jodis_addr = ""
jodis_timeout = "20s"
jodis_compatible = false

# Set datacenter of proxy.
proxy_datacenter = ""

# Set max number of alive sessions.
proxy_max_clients = 1000

# Set max offheap memory size. (0 to disable)
proxy_max_offheap_size = "1024mb"

# Set heap placeholder to reduce GC frequency.
proxy_heap_placeholder = "256mb"

# Proxy will ping backend redis (and clear 'MASTERDOWN' state) in a predefined interval. (0 to disable)
backend_ping_period = "5s"

# Set backend recv buffer size & timeout.
backend_recv_bufsize = "128kb"
backend_recv_timeout = "30s"

# Set backend send buffer & timeout.
backend_send_bufsize = "128kb"
backend_send_timeout = "30s"

# Set backend pipeline buffer size.
backend_max_pipeline = 1024

# Set backend never read replica groups, default is false
backend_primary_only = false

# Set backend parallel connections per server
backend_primary_parallel = 1
backend_replica_parallel = 1

# Set backend tcp keepalive period. (0 to disable)
backend_keepalive_period = "75s"

# Set number of databases of backend.
backend_number_databases = 16

# If there is no request from client for a long time, the connection will be closed. (0 to disable)
# Set session recv buffer size & timeout.
session_recv_bufsize = "128kb"
session_recv_timeout = "30m"

# Set session send buffer size & timeout.
session_send_bufsize = "64kb"
session_send_timeout = "30s"

# Make sure this is higher than the max number of requests for each pipeline request, or your client may be blocked.
# Set session pipeline buffer size.
session_max_pipeline = 10000

# Set session tcp keepalive period. (0 to disable)
session_keepalive_period = "75s"

# Set session to be sensitive to failures. Default is false, instead of closing socket, proxy will send an error response to client.
session_break_on_failure = false

# Set metrics server (such as http://localhost:28000), proxy will report json formatted metrics to specified server in a predefined period.
metrics_report_server = ""
metrics_report_period = "1s"

# Set influxdb server (such as http://localhost:8086), proxy will report metrics to influxdb.
metrics_report_influxdb_server = ""
metrics_report_influxdb_period = "1s"
metrics_report_influxdb_username = ""
metrics_report_influxdb_password = ""
metrics_report_influxdb_database = ""

# Set statsd server (such as localhost:8125), proxy will report metrics to statsd.
metrics_report_statsd_server = ""
metrics_report_statsd_period = "1s"
metrics_report_statsd_prefix = ""

4.3、redis.conf

protected-mode no
port 6379
maxmemory 10mb
dir "/codis"

5、启动zookeeper

可以使用外部的独立zookeeper，也可以使用docker容器，这里我使用codis-image提供的zookeeper容器：

./docker.sh zookeeper

6、启动dashboard

./docker.sh dashboard

7、启动proxy

./docker.sh proxy

8、启动codis-server（redis实例）

./docker.sh server

9、启动fe

./docker.sh fe

全部启动成功后如图：

10、访问fe

因为之前定义的fe映射端口为8080，所以直接：http://ip:8080即可打开fe界面

11、添加group/redis-server

12、直接点击rebalance all slots会自动初始化所有的slots

13、添加proxy

ip:21080

14、至止一个完整的基于docker的codis集群已经部署成功

转载请注明来源：http://www.16boke.com/article/detail/209

 Redis  Codis  Docker