Kubernetes v1.8.4 集群各组件加密认证功能的配置参考

来源：互联网发布：象过河软件免费版编辑：程序博客网时间：2024/06/15 15:06

记录 v1.8.4 的组件配置如下，经测试能完美地工作。

这里 kube-apiserver 并没有做 HA ，先这个测试通过了，后面再做。ha 方面，只需要加上 haproxy+keepalived进行配置，使其以 <vip>:443的方式访问即可。haproxy负责负载转发, keepalived负责监控haproxy的ha。
haproxy和keepalived可以部署在三台master中的两台上，这样注意一下端口问题，不要使用以master的api-server相冲突的端口即可。

好了不多说了，我们直接看下如何配置组件的加密认证功能，具体都添加了注释。

其中的 ca 证书，需要先在 master 中按照 master_ssl.cnf 进行配置生成 root 证书，即 ca.crt, ca.key，然后以他们去生成 server.crt, server.key。最后面将 ca.crt, ca.key 放到node节点中，继续为kubelet生成证书。

master_ssl.cnf

编辑 master_ssl.cnf，添加相关的配置

[alt_names]DNS.1 = kubernetesDNS.2 = kubernetes.defaultDNS.3 = kubernetes.default.svcDNS.4 = kubernetes.default.svc.k8s.xxx.com # specify domain name.DNS.5 = CDM1B12-209202200.wdds.com # apiserver hostnameIP.1 = 10.0.0.1 # kubernetes cluster ipIP.2 = 10.209.202.200  # kubernetes apiserver ip

在master上创建证书的脚本

function create_master_ca() {    echo "start create master ca ........."    openssl genrsa -out ca.key 2048    openssl req -x509 -new -nodes -key ca.key -subj "/CN=xxx.com" -days 5000 -out ca.crt    openssl genrsa -out server.key 2048    HN=`hostname`    echo "hostname is :$HN"    openssl req -new -key server.key -subj "/CN=$HN" -config master_ssl.cnf -out server.csr    openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt    openssl genrsa -out cs_client.key 2048    openssl req -new -key cs_client.key -subj "/CN=$HN" -out cs_client.csr    openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000    cp -f ca.* server.crt server.key cs_client.crt cs_client.key /etc/kubernetes/pki    cp -f /root/kube_package/ssl/kubeconfig /etc/kubernetes/kubeconfig    echo "end create master ca ........."}create_master_ca

在node上创建证书的脚本

#!/bin/bashcd `dirname $0`HN=`hostname`openssl genrsa -out kubelet_client.key 2048openssl req -new -key kubelet_client.key -subj "/CN=$HN" -out kubelet_client.csropenssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

kube-apiserver

apiVersion: v1kind: Podmetadata:  name: kube-apiserverspec:  hostNetwork: true  containers:  - name: kube-apiserver    image: 10.213.42.254:10500/root/hyperkube:v1.8.4-ceph    command:    - /bin/sh    - -c    - /hyperkube apiserver      --apiserver-count=1 #注意：这里没有添加ha      --allow-privileged=true      --etcd-prefix=/cd-dev02      --etcd-servers=http://10.209.202.200:2379,http://10.209.204.167:2379,http://10.209.204.199:2379      --admission-control=SecurityContextDeny,ServiceAccount,NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota      --insecure-bind-address=0.0.0.0      --insecure-port=11080 # 这个留着主要是为了兼容之前非安全使用之需。      --secure-port=443 # 在之前基础上添加这个安全端口即可。      --advertise-address=10.209.202.200      --service-cluster-ip-range=10.0.0.0/18      --tls-cert-file=/etc/kubernetes/pki/server.crt # 注意：这些文件都是挂载上去的，所以后面的 VolumeMounts 中需要特别指定。      --tls-private-key-file=/etc/kubernetes/pki/server.key      --client-ca-file=/etc/kubernetes/pki/ca.crt      --alsologtostderr=false      --logtostderr=true      --v=0      --log-dir=/var/log/kubernetes      --service-node-port-range=10000-12000      --storage-backend=etcd3      --storage-media-type=application/vnd.kubernetes.protobuf      --runtime-config=v1,extensions/v1beta1=true,extensions/v1beta1/ingress=true >> /var/log/kubernetes/kube-apiserver.log 2>&1    ports:        - containerPort: 443          hostPort: 443          name: https        - containerPort: 7080          hostPort: 7080          name: http        - containerPort: 11080          hostPort: 11080          name: local        - containerPort: 6443          hostPort: 6443          name: seport        volumeMounts:        - mountPath: /etc/kubernetes          name: pki          readOnly: true        - mountPath: /var/log          name: logpath        - mountPath: /etc/localtime          name: localtime      volumes:      - hostPath:          path: /etc/kubernetes        name: pki      - hostPath:          path: /var/log        name: logpath      - hostPath:          path: /etc/localtime        name: localtime

kube-controller-manager

apiVersion: v1kind: Podmetadata:  name: kube-controller-managerspec:  hostNetwork: true  containers:  - name: kube-controller-manager    image: 10.213.42.254:10500/root/hyperkube:v1.8.4-ceph    command:    - /bin/sh    - -c    - /hyperkube controller-manager      --v=0      --logtostderr=true      --log-dir=/var/log/kubernetes      --alsologtostderr=false      --root-ca-file=/etc/kubernetes/pki/ca.crt      --service-account-private-key-file=/etc/kubernetes/pki/server.key      --kubeconfig=/etc/kubernetes/kubeconfig #这个yaml文件中本来要指定 --master=https://vip:443 的，但后来版本不支持了，都移到kubeconfig中进行配置了      --leader-elect=true >> /var/log/kubernetes/kube-controller-manager.log 2>&1    ports:    - containerPort: 10252      hostPort: 10252      name: local    volumeMounts:    - mountPath: /etc/kubernetes      name: pki      readOnly: true    - mountPath: /var/log      name: logpath    - mountPath: /sbin/modprobe      name: modprobe      readOnly: true    - mountPath: /lib/modules      name: modules      readOnly: true    - mountPath: /dev      name: devices  volumes:  - hostPath:      path: /etc/kubernetes    name: pki  - hostPath:      path: /var/log    name: logpath  - hostPath:      path: /sbin/modprobe    name: modprobe  - hostPath:      path: /lib/modules    name: modules  - hostPath:      path: /dev    name: devices

kube-scheduler

apiVersion: v1kind: Podmetadata:  name: kube-schedulerspec:  hostNetwork: true  containers:  - name: kube-scheduler    image: 10.213.42.254:10500/root/hyperkube:v1.8.4-ceph    command:    - /bin/sh    - -c    - /hyperkube scheduler      --kubeconfig=/etc/kubernetes/kubeconfig #这个yaml文件中本来要指定 --master=https://vip:443 的，但后来版本不支持了，都移到kubeconfig中进行配置了      --v=0      --logtostderr=true      --alsologtostderr=false      --log-dir=/var/log/kubenetes      --leader-elect=true >> /var/log/kubernetes/kube-scheduler.log 2>&1    ports:    - containerPort: 10251      hostPort: 10251      name: local    volumeMounts:    - mountPath: /etc/kubernetes      name: pki      readOnly: true    - mountPath: /var/log      name: logpath    - mountPath: /etc/localtime      name: localtime  volumes:    - hostPath:        path: /etc/kubernetes      name: pki    - hostPath:        path: /var/log      name: logpath    - hostPath:        path: /etc/localtime      name: localtime

master中的 /etc/kubernetes/kubeconfig

apiVersion: v1kind: Configusers:- name: controllermanager  user:    client-certificate: /etc/kubernetes/pki/cs_client.crt    client-key: /etc/kubernetes/pki/cs_client.keyclusters:- name: local  cluster:    server: https://10.209.202.200:443 #本来要在controller-manager和scheduler中的yaml文件中指定 --master=https://vip:443 的，但后来版本不支持了，都移到这里进行配置了，并且 https:// 这几个字不能去掉。    certificate-authority: /etc/kubernetes/pki/ca.crtcontexts:- context:    cluster: local    user: controllermanager  name: my-contextcurrent-context: my-context

kubelet.service

[Unit]Description=Kubernetes Kubelet ServerDocumentation=http://kubernetes.io/docs/admin/kubelet/After=docker.service[Service]WorkingDirectory=/var/lib/kubeletEnvironmentFile=-/etc/default/kube-defaultEnvironmentFile=-/etc/default/kubeletExecStart=/bin/sh -c '/usr/local/bin/kubelet \        --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin \        --hostname-override=10.209.228.18 \        --kubeconfig=/etc/kubernetes/kubeconfig \ #这个文件中本来要指定 --master=https://vip:443 的，但后来版本不支持了，都移到kubeconfig中进行配置了        --pod-manifest-path=/etc/kubernetes/manifests \        --require-kubeconfig=true \        --logtostderr=true \        --pod-infra-container-image=10.213.42.254:10500/pause:3.0 \        --cluster-dns=10.0.0.10 \        --cluster-domain=k8s.wanda.com \        --max-pods=110 \        --cgroup-driver=cgroupfs \        --fail-swap-on=false \        --runtime-cgroups=/systemd/system.slice \        --kubelet-cgroups=/systemd/system.slice \        --allow-privileged=true -v=0 >> /var/log/kubernetes/kubelet.log 2>&1'Restart=alwaysStartLimitInterval=0RestartSec=10[Install]WantedBy=multi-user.target

node 节点中的 /etc/kubernetes/kubeconfig

apiVersion: v1kind: Configusers:- name: kubelet  user:    client-certificate: /etc/kubernetes/ssl/kubelet_client.crt    client-key: /etc/kubernetes/ssl/kubelet_client.keyclusters:- name: local  cluster:    server: https://10.209.202.200:443 #本来要在kubelet.service文件中指定 --api_servers=https://vip:443 的，但后来版本不支持了，都移到这里进行配置了，而且将--api-server 改成 server了，并且 https:// 这几个字不能去掉。    certificate-authority: /etc/kubernetes/ssl/ca.crtcontexts:- context:    cluster: local    user: kubelet  name: my-contextcurrent-context: my-context

kube-proxy.yaml

apiVersion: v1kind: Podmetadata:  name: kube-proxyspec:  hostNetwork: true  containers:  - name: kube-proxy    image: 10.213.42.254:10500/root/hyperkube:v1.8.4-ceph    command:    - /bin/sh    - -c    - /hyperkube proxy      --logtostderr=true      --proxy-mode=iptables      --master=https://10.209.202.200:443 #感觉这个可以不要，因为我已经在kubeconfig中用server指定了，即server来代替--api-servers。但这个server是否也能替代这里的--master？？后面要测试一下这个。      --kubeconfig=/etc/kubernetes/kubeconfig      -v=4      --conntrack-tcp-timeout-established=1200s  >> /var/log/kubernetes/kube-proxy.log 2>&1    securityContext:      privileged: true    volumeMounts:    - mountPath: /etc/kubernetes      name: pki      readOnly: true    - mountPath: /var/log      name: logpath    - mountPath: /etc/localtime      name: localtime  volumes:  - hostPath:      path: /etc/kubernetes    name: pki  - hostPath:      path: /var/log    name: logpath  - hostPath:      path: /etc/localtime    name: localtime

阅读全文

0 0