kubernetes 1.5安装 Redis 3.2.6 Cluster StatefulSet

来源：互联网发布：gta5pc男角色捏脸数据编辑：程序博客网时间：2024/06/03 21:38

听说Redis 3以后安装集群比较简单，几个命令就可以完全搞定，在自己的电脑上面安装试了下，果然比较爽。

今天准备把Redis 3的集群安装在K8s上面，通常的做法都是在github上面查找是不是有人已经做过，果然很多。下面的代码基本上来自于github上面。不过就算用这个面的代码，在自己部署的时候也存在不少的问题。因为github上面的都是大神，很比比较小的问题不会写的很清楚。

所以：如果这个文档（blog）如何解决不了问题，就到github上面去查找别的方法。

1>>>>>>>>>>>镜像说明

使用的镜像名: ausov/redis:3.2-alpine

先使用下面命令下载镜像:

docker pull ausov/redis:3.2-alpine

这个镜像如果无法下载，则需要自己手动build一下.

这个是编译需要使用的文件

[root@centos-master alpine]# pwd/root/k8s-redis/3.2/alpine[root@centos-master alpine]# lsdocker-entrypoint.sh  Dockerfile  redis.conf

cat docker-entrypoint.sh

#!/bin/shset -eox pipefail#shopt -s nullglobREDIS_CONF=${REDIS_CONF:-"/opt/k8s-redis/redis.conf"}if [ "${1#-}" != "$1" ] || [ "${1%.conf}" != "$1" ]; then  set -- redis-server "$@"fiif [ "$1" = 'redis-server' ] && [ -n "$SLAVEOF" ] && [ -z "$SENTINEL" ]; then  echo "Starting Redis replica"  set -- $@ "$REDIS_CONF" --slaveof "$SLAVEOF" 6379elif [ "$1" = 'redis-server' ] && [ -n "$SENTINEL" ]; then  echo "Starting Redis sentinel"  while true; do    redis-cli -h $SENTINEL INFO    if [[ "$?" == "0" ]]; then      break    fi    echo "Connecting to master failed.  Waiting..."    sleep 10  done  echo "sentinel monitor primary $SENTINEL 6379 2" >> "$REDIS_CONF"  echo "sentinel down-after-milliseconds primary 5000" >> "$REDIS_CONF"  echo "sentinel failover-timeout primary 10000" >> "$REDIS_CONF"  echo "sentinel parallel-syncs primary 1" >> "$REDIS_CONF"  set -- $@ "$REDIS_CONF" --port 26379 --sentinel --protected-mode noelif [ "$1" = 'redis-server' ]; then    echo "Starting Redis master"  set -- $@ "$REDIS_CONF"fiexec "$@"

这个文件内容太多，我们在这里看下可用的就OK。

cat redis.conf | grep -Ev '(^#|^$)'

protected-mode noport 6379tcp-backlog 511timeout 0tcp-keepalive 300daemonize nosupervised nopidfile /var/run/redis_6379.pidloglevel noticelogfile ""databases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yesdbfilename dump.rdbdir ./slave-serve-stale-data yesslave-read-only yesrepl-diskless-sync norepl-diskless-sync-delay 5repl-disable-tcp-nodelay noslave-priority 100appendonly yesappendfilename "appendonly.aof"appendfsync everysecno-appendfsync-on-rewrite noauto-aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mbaof-load-truncated yeslua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128latency-monitor-threshold 0notify-keyspace-events ""hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-size -2list-compress-depth 0set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64hll-sparse-max-bytes 3000activerehashing yesclient-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yes

cat Dockerfile

FROM redis:3.2-alpineCOPY ["redis.conf", "/opt/k8s-redis/"]COPY ["docker-entrypoint.sh", "/usr/local/bin/"]

以上就是编译需要使用到的文件。cd到这个目录，执行下面的命令。

docker build -t "ausov/redis:3.2-alpine" .

镜像编译完成以后，需要确保所有的节点都有这个镜像。

2>>>>>>>>>>>部署安装

部署安装我们需要使用三个文件，分别是:

primary.yml secondary.yml sentinel.yml

下面是文件内容:

[root@centos-master example]# cat primary.yml apiVersion: v1kind: Servicemetadata:  name: redis-primary  labels:    app: redis-primaryspec:  ports:  - port: 6379    name: redis-primary  clusterIP: None  selector:    app: redis-primary---apiVersion: apps/v1beta1kind: StatefulSetmetadata:  name: redis-primaryspec:  serviceName: redis-primary  replicas: 1  template:    metadata:      labels:        app: redis-primary    spec:      terminationGracePeriodSeconds: 10      containers:      - name: redis-primary        image: ausov/redis:3.2-alpine        imagePullPolicy: IfNotPresent        ports:        - containerPort: 6379          name: redis-primary        volumeMounts:        - name: data          mountPath: /data  volumeClaimTemplates:  - metadata:      name: data      annotations:        volume.alpha.kubernetes.io/storage-class: anything    spec:      accessModes: [ "ReadWriteOnce" ]      resources:        requests:          storage: 5Gi

[root@centos-master example]# cat secondary.yml apiVersion: v1kind: Servicemetadata:  name: redis-secondary  labels:    app: redis-secondaryspec:  ports:  - port: 6379    name: redis-secondary  clusterIP: None  selector:    app: redis-secondary---apiVersion: apps/v1beta1kind: StatefulSetmetadata:  name: redis-secondaryspec:  serviceName: redis-secondary  replicas: 2  template:    metadata:      labels:        app: redis-secondary    spec:      terminationGracePeriodSeconds: 10      containers:      - name: redis-secondary        image: ausov/redis:3.2-alpine        imagePullPolicy: IfNotPresent        env:        - name: SLAVEOF          value: redis-primary-0.redis-primary        ports:        - containerPort: 6379          name: redis-secondary        volumeMounts:        - name: data          mountPath: /data  volumeClaimTemplates:  - metadata:      name: data      annotations:        volume.alpha.kubernetes.io/storage-class: anything    spec:      accessModes: [ "ReadWriteOnce" ]      resources:        requests:          storage: 5Gi

[root@centos-master example]# cat sentinel.yml apiVersion: v1kind: Servicemetadata:  name: redis-sentinel  labels:    app: redis-sentinelspec:  ports:  - port: 26379    name: redis-sentinel  clusterIP: None  selector:    app: redis-sentinel---apiVersion: apps/v1beta1kind: StatefulSetmetadata:  name: redis-sentinelspec:  serviceName: redis-sentinel  replicas: 3  template:    metadata:      labels:        app: redis-sentinel    spec:      terminationGracePeriodSeconds: 10      containers:      - name: redis-sentinel        image: ausov/redis:3.2-alpine        imagePullPolicy: IfNotPresent        env:        - name: SENTINEL          value: redis-primary-0.redis-primary        ports:        - containerPort: 26379          name: redis-sentinel

三个文件都存在以后:

 kubectl create -f primary.yml  -f secondary.yml  -f  sentinel.yml

创建完成后，查看一下pod

kubectl get pod

会得到以下信息:

NAME                                READY     STATUS    RESTARTS   AGE

redis-primary-0                     1/1       Running   0          2dredis-secondary-0                   1/1       Running   0          2dredis-secondary-1                   1/1       Running   0          2dredis-sentinel-0                    1/1       Running   0          2dredis-sentinel-1                    1/1       Running   0          2dredis-sentinel-2                    1/1       Running   0          2d

下面是查看StatefulSet的信息：

[root@centos-master example]# kubectl get statefulsetNAME              DESIRED   CURRENT   AGEredis-primary     1         1         2dredis-secondary   2         2         2dredis-sentinel    3         3         2d

必须要确保DESIRED和CURRENT的值是一样的。

查看持久存储的信息:

[root@centos-master example]# kubectl get pvc

NAME                                       STATUS    VOLUME                                     CAPACITY   ACCESSMODES   AGE

data-redis-primary-0                       Bound     pvc-94534255-d614-11e6-955d-1866daed1b04   5Gi        RWO           2ddata-redis-secondary-0                     Bound     pvc-9a8f2080-d614-11e6-955d-1866daed1b04   5Gi        RWO           2ddata-redis-secondary-1                     Bound     pvc-9a9029b4-d614-11e6-955d-1866daed1b04   5Gi        RWO           2d

我们在注意这里的STATUS，是Bound。

如果不是，用命令

describe pvc data-redis-primary-0

如果出现下面的问题：

cannot find volume plugin for alpha provisioning

则需要修改文件:

/etc/kubernetes/manifests/kube-controller-manager.json

增加参数: --enable-hostpath-provisioner=true

以下是截取了这个文件的一部分:

        "name": "kube-controller-manager",        "image": "gcr.io/google_containers/kube-controller-manager-amd64:v1.5.1",        "command": [          "kube-controller-manager",          "--address=127.0.0.1",          "--leader-elect",          "--master=127.0.0.1:8080",          "--cluster-name=kubernetes",          "--root-ca-file=/etc/kubernetes/pki/ca.pem",          "--service-account-private-key-file=/etc/kubernetes/pki/apiserver-key.pem",          "--cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem",          "--cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem",          "--insecure-experimental-approve-all-kubelet-csrs-for-group=system:kubelet-bootstrap",          "--allocate-node-cidrs=true",          "--enable-hostpath-provisioner=true",          "--cluster-cidr=10.245.0.0/16"        ],

修改保存，重启一下docker或kubelet.

通过上面的配置，Redis基本就配置完成:

这两个WARNING暂不影响redis使用，解决方法过两天再写上。

11:M 04 Jan 09:45:23.515 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
11:M 04 Jan 09:45:23.515 # Server started, Redis version 3.2.1
11:M 04 Jan 09:45:23.515 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
11:M 04 Jan 09:45:23.516 * The server is now ready to accept connections on port 6379
OK

0 0