在CentOS6.7下安装私有Docker Registry

前言

---

由于目前大多数的镜像下载网站都是在国外站点，国内用户在下镜像的时候就时常遇到下载速度超慢的情况；而且在将Docker应用到生产环境的过程中，还会面临下载过来的Docker镜像是否安全的问题。因些在生产环境下搭建私有的Docker Registry，就是一个一举二得的方案了。

但目前国内的大多数企业使用的发行版，为Redhat体系，其中又以Centos环境居多。而网上的资料和出版的书籍在介绍Docker的时候都是以ubuntu为基础进行讲解和配置，而CentOS6.x以上版本访问私有的 Docker
Registry 都需要SSL的支持。因此使用CentOS来搭建此服务最大的难点是配置和使用SSL，并让做为反向代理的nginx支持SSL传输和认证。

约定

• 服务器IP：172.20.31.201
• 服务器域名：registry.cmzsteven.com
  #https的SSL证书生成时不能用IP地址因此这里随意设置一个域名即可，不一定为真实域名。
• 客户端IP：172.20.31.200
• 代理服务器：nignx (反向代理)

一、配置流程

为方便大家从总体上掌握服务搭建，在这里简单叙述一下配置流程：

1. 在配置端配置SSL：生成密钥、根证书，为nginx服务生成SSL密钥，最后私有CA根据请求完成对nginx证书的签发；
2. 编译、安装nginx，并配置为反向代理。监听https的433，对客户端发出的针对433端口的请求转发到registry服务的5000端口；
3. 安装Docker的registry镜像并运行容器，监听来自nginx转发过来的请求；
4. 在客户端完成SSL证书的添加，并完成上传及下载镜像的操作，完成测试。

二、在Registry服务器上配置SSL

1、安装相关依赖包

yum install -y gcc pcre-devel pcre-static openssl openssl-devel httpd-tools

注：

• gcc 是编译器；
• pcre-devel pcre-static 是编译nginx时需要的依赖包；
• openssl openssl-devel 用于生成SSL根证书及密钥并进行nignx证书的签发；
• httpd-tools 用来生成登录https服务的帐号及密码。

2、添加域名到本地/etc/hosts

echo "172.20.31.201 registry.cmzsteven.com" >> /etc/hosts

3、生成根密钥

cd /etc/pki/CArm -rf cacert.pem index.txt index.txt.attr index.txt.old serial serial.oldopenssl genrsa -out private/cakey.pem 2048

运行结果：

Generating RSA private key, 2048 bit long modulus...................................................+++....................................................................................+++e is 65537 (0x10001)

4、生成根证书

cd /etc/pki/CAopenssl req -new -x509 -key private/cakey.pem -out cacert.pem

运行结果:

You are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [XX]:CNState or Province Name (full name) []:liaoningLocality Name (eg, city) [Default City]:shenyangOrganization Name (eg, company) [Default Company Ltd]:cmzstevenstudioOrganizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) []:registry.cmzsteven.comEmail Address []:

在运行过程中会提示一些信息用于生成私有的CA根证书。需要特别注意的是倒数第二行“Common Name”，这里需要输入的是事先准备好的服务器域名
生成的证书存放的位置：/etc/pki/CA/cacert.pem

5、为Nginx生成SSL密钥

mkdir -p /etc/nginx/sslcd /etc/nginx/sslopenssl genrsa -out nginx.key 2048

运行结果:

Generating RSA private key, 2048 bit long modulus...............................................................+++......................................+++e is 65537 (0x10001)

*CA中心服务与要申请证书的Nginx服务应该运行在同一台服务器上，否则应该是在需要运行Nginx服务的服务器上生成证书。

6、为Nginx生成证书签署请求

openssl req -new -key nginx.key -out nginx.csr

运行结果：

Generating RSA private key, 2048 bit long modulus...............................................................+++......................................+++e is 65537 (0x10001)[root@DockerRegistry ssl]# openssl req -new -key nginx.key -out nginx.csrYou are about to be asked to enter information that will be incorporatedinto your certificate request.What you are about to enter is what is called a Distinguished Name or a DN.There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [XX]:CNState or Province Name (full name) []:liaoningLocality Name (eg, city) [Default City]:shenyangOrganization Name (eg, company) [Default Company Ltd]:cmzstevenstudioOrganizational Unit Name (eg, section) []:Common Name (eg, your name or your server's hostname) []:registry.cmzsteven.comEmail Address []:Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:An optional company name []:

生成过程与之前生成CA根证书的过程差不多。
“Common Name”需要填写事先准备好的域名。
最后多出的几行”challenge passwd”和”optional company name”可以为空。

7、使用私有CA根据Nginx的请求来签发证书

touch /etc/pki/CA/index.txttouch /etc/pki/CA/serialecho 00 > /etc/pki/CA/serialopenssl ca -in nginx.csr -out nginx.crt

运行结果：

Using configuration from /etc/pki/tls/openssl.cnfCheck that the request matches the signatureSignature okCertificate Details:        Serial Number: 0 (0x0)        Validity            Not Before: Oct 24 03:13:32 2015 GMT            Not After : Oct 23 03:13:32 2016 GMT        Subject:            countryName               = CN            stateOrProvinceName       = liaoning            organizationName          = cmzstevenstudio            commonName                = registry.cmzsteven.com        X509v3 extensions:            X509v3 Basic Constraints:                CA:FALSE            Netscape Comment:                OpenSSL Generated Certificate            X509v3 Subject Key Identifier:                59:6D:08:34:57:47:F4:5E:28:62:D5:60:3C:CF:37:10:45:70:53:66            X509v3 Authority Key Identifier:                keyid:AC:06:A4:97:32:75:46:13:BC:15:78:EC:F9:35:19:B9:22:DE:0D:B7Certificate is to be certified until Oct 23 03:13:32 2016 GMT (365 days)Sign the certificate? [y/n]:y1 out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base Updated

在提示需要输入”[y/n]”时，都输入”y”就可以了。

三、在服务器上编译安装Nginx，并配置为反向代理

1、下载Nginx源文件

cd /usr/local/srcyum install -y wgetwget http://nginx.org/download/nginx-1.8.0.tar.gz

2、编译安装Nginx

 tar -zxvf nginx-1.8.0.tar.gz cd nginx-1.8.0 ./configure --prefix=/usr/local/nginx --with-pcre --with-http_stub_status_module --with-http_ssl_module --with-http_addition_module   --with-http_realip_module --with-http_flv_modulemakemake install

3、编译Nginx配置文件

cd cd /usr/local/nginx/confvim nginx.conf

内容如下：

user nobody nobody;worker_processes 2;error_log /usr/local/nginx/logs/nginx_error.log crit;pid /usr/local/nginx/logs/nginx.pid;worker_rlimit_nofile 51200;events{    use epoll;    worker_connections 6000;}http{    include mime.types;    default_type application/octet-stream;    server_names_hash_bucket_size 3526;        server_names_hash_max_size 4096;    log_format combined_realip '$remote_addr $http_x_forwarded_for [$time_local]' '$host "$request_uri" $status' '"$http_referer" "$http_user_agent"';    sendfile on;    tcp_nopush on;    keepalive_timeout 30;    client_header_timeout 3m;    client_body_timeout 3m;    send_timeout 3m;    connection_pool_size 256;    client_header_buffer_size 1k;    large_client_header_buffers 8 4k;    request_pool_size 4k;    output_buffers 4 32k;    postpone_output 1460;    client_max_body_size 10m;    client_body_buffer_size 256k;    client_body_temp_path /usr/local/nginx/client_body_temp;    proxy_temp_path /usr/local/nginx/proxy_temp;    fastcgi_temp_path /usr/local/nginx/fastcgi_temp;    fastcgi_intercept_errors on;    tcp_nodelay on;    gzip on;    gzip_min_length 1k;    gzip_buffers 4 8k;    gzip_comp_level 5;    gzip_http_version 1.1;    gzip_types text/plain application/x-javascript text/css text/htm application/xml;    upstream registry {        server 127.0.0.1:5000;    }  server {    listen 443;    server_name  10.0.0.201;#服务器的IP    ssl  on;    ssl_certificate /etc/nginx/ssl/nginx.crt;    ssl_certificate_key /etc/nginx/ssl/nginx.key;       client_max_body_size 0;    chunked_transfer_encoding on;    location / {      auth_basic "registry";      auth_basic_user_file /usr/local/nginx/conf/.htpasswd;      root   html;      index index.html index.htm;      proxy_pass                  http://registry;      proxy_set_header  Host           $http_host;      proxy_set_header  X-Real-IP      $remote_addr;      proxy_set_header  Authorization  "";      client_body_buffer_size     128k;      proxy_connect_timeout       90;      proxy_send_timeout          90;      proxy_read_timeout          90;      proxy_temp_file_write_size  64k;    }    location /_ping {      auth_basic off;      proxy_pass http://registry;    }    location /v1/_ping {      auth_basic off;      proxy_pass http://registry;    }  }}

验证配置是否正确：

[root@DockerRegistry nginx]# sbin/nginx -tnginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is oknginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

4、启动Nginx服务
4.1、新建启动角本

touch /etc/init.d/nginxchmod 755 /etc/init.d/nginxvim /etc/init.d/nginx

角本内容如下：

#!/bin/bash# chkconfig: - 30 21# description: http service.# Source Function Library. /etc/init.d/functions# Nginx SettingsNGINX_SBIN="/usr/local/nginx/sbin/nginx"NGINX_CONF="/usr/local/nginx/conf/nginx.conf"NGINX_PID="/usr/local/nginx/logs/nginx.pid"RETVAL=0prog="Nginx"start() {        echo -n $"Starting $prog: "        mkdir -p /dev/shm/nginx_temp        daemon $NGINX_SBIN -c $NGINX_CONF        RETVAL=$?        echo        return $RETVAL}stop() {        echo -n $"Stopping $prog: "        killproc -p $NGINX_PID $NGINX_SBIN -TERM        rm -rf /dev/shm/nginx_temp        RETVAL=$?        echo        return $RETVAL}reload(){        echo -n $"Reloading $prog: "        killproc -p $NGINX_PID $NGINX_SBIN -HUP        RETVAL=$?        echo        return $RETVAL}restart(){        stop        start}configtest(){    $NGINX_SBIN -c $NGINX_CONF -t    return 0}case "$1" in  start)        start        ;;  stop)        stop        ;;  reload)        reload        ;;  restart)        restart        ;;  configtest)        configtest        ;;  *)        echo $"Usage: $0 {start|stop|reload|restart|configtest}"        RETVAL=1esacexit $RETVAL

4.2、启动服务

chkconfig nginx onservice nginx start

查看服务是否启动：

[root@DockerRegistry nginx]# ps aux |grep nginx|grep -v greproot     13873  0.0  0.1  44612  1184 ?        Ss   11:56   0:00 nginx: master process sbin/nginxnobody   13874  0.0  0.3  47104  3948 ?        S    11:56   0:00 nginx: worker processnobody   13875  0.0  0.3  47104  3852 ?        S    11:56   0:00 nginx: worker process

5、生成登录https服务的账户和密码

htpasswd -cb /usr/local/nginx/conf/.htpasswd docker 147258

注:参数c是生成数据库文件，如果再添加帐户的话就不需要加空上参数了；参数b是在命令中带有账户密码（在生产环境中不推荐使用。

四、服务器上安装配置Docker Registry服务

1、安装Docker

yum -y install docker-io

2、修改Docker配置文件

echo "DOCKER_OPTS=\"--insecure-registry docker.yy.com --tlsverify --tlscacert /etc/pki/CA/cacert.pem\"" >> /etc/sysconfig/docker

3、启动Docker服务

chkconfig docker onservice docker start

4、下载registry镜像

docker pull registry

5、运行registry镜像

mkdir /docker_registry  #在宿主机上新建用于存放镜像的文件夹docker run -d -p 5000:5000 -v /docker_registry:/tmp/registry --name=docker_registry registry

命令详解：

• -p 5000：5000 将本地5000端口映射到容器的5000端口上；
• -v 将本地的文件夹/docker_registry映射到容器存放镜像文件的/tmp/registry文件夹；
• –name 将生成的容器命名为：docker_registry。

查看生成的容器：

[root@DockerRegistry nginx]# docker ps -aCONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES2185389323f5        registry            "docker-registry"   14 seconds ago      Up 12 seconds       0.0.0.0:5000->5000/tcp   docker_registry

6、将容器docker_registry设置为开自自启动

echo "docker start 2185389323f5" >> /etc/rc.local

7、测试通过Nginx访问registry服务

[root@DockerRegistry nginx]# curl -i -k -u docker:147258  https://registry.cmzsteven.com:443HTTP/1.1 200 OKServer: nginx/1.8.0Date: Sat, 24 Oct 2015 05:24:22 GMTContent-Type: application/jsonContent-Length: 28Connection: keep-aliveExpires: -1Pragma: no-cacheCache-Control: no-cache"\"docker-registry server\""

如果测试结果出现“HTTP/1.1 200 OK”和 “\”docker-registry server\”” 说明registry服务器已经搭建成功。

五、客户端配置SSL

1、在客户端上添加registry的域名解析

echo "172.20.31.201 registry.cmzsteven.com" >> /etc/hosts

2、将registry服务器上的根证书内容复制到客户端ca-certificates.crt里
服务器查看根证书内容：

[root@DockerRegistry nginx]# cat /etc/pki/CA/cacert.pem

根证书内容样例：

-----BEGIN CERTIFICATE-----MIIDrzCCApegAwIBAgIJAOIsiUqgCNd+MA0GCSqGSIb3DQEBBQUAMG4xCzAJBgNVBAYTAkNOMREwDwYDVQQIDAhsaWFvbmluZzERMA8GA1UEBwwIc2hlbnlhbmcxGDAWBgNVBAoMD2NtenN0ZXZlbnN0dWRpbzEfMB0GA1UEAwwWcmVnaXN0cnkuY216c3RldmVuLmNvbTAeFw0xNTEwMjQwMjUwNTlaFw0xNTExMjMwMjUwNTlaMG4xCzAJBgNVBAYTAkNOMREwDwYDVQQIDAhsaWFvbmluZzERMA8GA1UEBwwIc2hlbnlhbmcxGDAWBgNVBAoMD2NtenN0ZXZlbnN0dWRpbzEfMB0GA1UEAwwWcmVnaXN0cnkuY216c3RldmVuLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALCGouLPUJpFQXMLZlG1CORskWXd9HK2s1Vd2lwb6ufePZ7WMBtS59cdQmcXHZSdSGzwmZtUY+M647x4Har9T+4RGiBvD2AvFqQ9SxziKzI4u3BTROmeEJocsEaW6LXDqlsh9tYfVAbR10/392WvGNTIgL5iPd+depEcilV7M7i3vd3sTmJ3JCDm6yATTs9B5vNeBlYyuQP6rQjBla2+KOZeUJ8u4VCl8FZWzuepQBMMwF8yOujBFP4xuhumzeg7bqYnxQeeMjyxK2jZc+cRLNdJ5qi66MQJksRt5F2EU402P80dbaJH7Aqc2/rEAbjLCTsPuI3VBUVZpec69ZmtudMCAwEAAaNQME4wHQYDVR0OBBYEFKwGpJcydUYTvBV47Pk1Gbki3g23MB8GA1UdIwQYMBaAFKwGpJcydUYTvBV47Pk1Gbki3g23MAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEFBQADggEBAIWcx606kPeA6CA4KcPZgDULMu5ZmOwHoXCGKez8EbRI5Vc3IgteFXalRf4Z6PrOh/u4qY0LxgeSOcD5Ws6qA0hAexRqrxsH2mYcbADMwnL5xUp22y5HxuZ2NpRV4rEe8s8vUEgrJECJgfbyMKXPHGwLCnIYu0YxYParYkfLHd4nE3h0ZiXhry6p3k2//0WwLQdW8lnRO3mTNOtfJVUzoNtIOCK/kkKTMqPX+U+zb0j65mlTLqTkhebtETiLd6XwO5QsNXU6iQ5iJgSrAlQVX/2VMgOXsqX6QJBetYiemfGqaZIpppcffUpDdP8WNv1t4nLac2gEQZjNo5GY5APCs2A=-----END CERTIFICATE-----

上你搭建的服务器上的根证书的内容复制到

touch /etc/pki/tls/certs/ca-certificates.crtvim /etc/pki/tls/certs/ca-certificates.crt

3、测试访问服务器registry

[root@localhost ~]# curl -i -k -u docker:147258 https://registry.cmzsteven.comHTTP/1.1 200 OKServer: nginx/1.8.0Date: Sat, 24 Oct 2015 05:46:24 GMTContent-Type: application/jsonContent-Length: 28Connection: keep-aliveExpires: -1Pragma: no-cacheCache-Control: no-cache"\"docker-registry server\""

OK!

4、客户端通过Docker登录registry服务

docker login -u docker -p 147258 https://registry.cmzsteven.com

当client端 docker login出现 x509: certificate signed by unknown authority 错误时的解决方法：

#重命名根证书mv /etc/pki/tls/certs/ca-certificates.crt /etc/pki/tls/certs/ca-certificates.crt.bak#重启docker服务! service docker restart

六、客户端上传镜像

1、下载centos镜像用于测试*

docker pull centos

下载后查看镜像：

[root@localhost ~]# docker imagesREPOSITORY                      TAG                 IMAGE ID            CREATED             VIRTUAL SIZEcentos                          latest              0f73ae75014f        6 weeks ago         172.3 MB

2、给下载的镜像打个私有仓库的tag

docker tag centos:latest registry.cmzsteven.com/centos:latest

查看镜像：

[root@localhost ~]# docker imagesREPOSITORY                      TAG                 IMAGE ID            CREATED             VIRTUAL SIZEregistry.cmzsteven.com/centos   latest                0f73ae75014f        6 weeks ago         172.3 MBcentos                          latest              0f73ae75014f        6 weeks ago         172.3 MB

3、上传镜像到registry服务器

[root@localhost ~]# docker push registry.cmzsteven.com/centos:latestThe push refers to a repository [registry.cmzsteven.com/centos] (len: 1)Sending image listPushing repository registry.cmzsteven.com/centos (1 tags)47d44cb6f252: Image successfully pushedf6f39725d938: Image successfully pushedf9a8cbc8dd13: Image successfully pushedf37e6a610a37: Image successfully pushed0f73ae75014f: Image successfully pushedPushing tag for rev [0f73ae75014f] on {https://registry.cmzsteven.com/v1/repositories/centos/tags/latest}

七、客户端下载镜像

1、从registry服务器下载镜像
为了测试先删除之前打过tag的镜像：

[root@localhost ~]# docker rmi registry.cmzsteven.com/centos:latestUntagged: registry.cmzsteven.com/centos:latest

下载镜像：

[root@localhost ~]# docker pull registry.cmzsteven.com/centos:latestPulling repository registry.cmzsteven.com/centos0f73ae75014f: Download complete47d44cb6f252: Download completef6f39725d938: Download completef9a8cbc8dd13: Download completef37e6a610a37: Download completeStatus: Image is up to date for registry.cmzsteven.com/centos:latest

2、查看下载的镜像

[root@localhost ~]# docker imagesREPOSITORY                      TAG                 IMAGE ID            CREATED             VIRTUAL SIZEcentos                          latest              0f73ae75014f        6 weeks ago         172.3 MBregistry.cmzsteven.com/centos   latest              0f73ae75014f        6 weeks ago         172.3 MB