使用 harbor 搭建 docker 私有镜像仓库

harbor 是由 vmware 中国团队开发并开源的企业级 docker registry 服务，是对官方开源的 Docker Registry 的扩展，增加了一些企业需要的功能，如安全、复制和管理。harbor 主要用于搭建私有 registry，提供了企业需要的安全和控制功能。同时，它也帮助减少带宽使用量，这对提供生产能力和性能有帮助。

harbor 的强大之处在于，同时提供了 registry 的存储功能、认证功能、web ui 浏览和管理功能，且搭建相对简单。

本文搭建环境：

$ cat /etc/redhat-releaseCentOS Linux release 7.1.1503 (Core) $ uname -srvmpioLinux 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Sep 15 15:05:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux$ ip addr show | grep eth0 | grep inetinet 172.20.30.35/24 brd 172.20.30.255 scope global dynamic eth0

搭建目标是：

能通过 http://172.20.30.35:10080 打开 web ui；

能通过 172.20.30.35:10080 push/pull docker 镜像。

而 harbor 的默认使用的端口是 80，我们要改变这个端口到 10080，因为目标主机的 80 端口在外部无法访问。

Harbor 的安装方法有两种：

1. 从源码安装 -- 这通过一个完全的构建过程，且需要你的主机能连接外网
2. 预构建安装包 -- 这比较节省时间（不需要构建），同时允许在一台没有联网的主机上搭建

这篇文章同时介绍这两种安装方式。

搭建必备条件

harbor 是以若干个 docker 容器化应用部署的，所以可以安装在任何支持 docker 的 linux 机器上。目标主机需要安装 python 2.7或以上, docker 1.10或以上，docker-compose 1.6.0 或以上。

从源码安装

1、获取源码

$ git clone https://github.com/vmware/harbor

然后

$ cd harbor/Deploy

注意：除非明确说明，否则后续所有操作都在这个目录中进行。

2、配置

主要配置下面几个文件（改动的行用 ** 标记）：

2.1

$ vim templates/ui/app.confappname = registryrunmode = dev[lang]types = en-US|zh-CNnames = en-US|zh-CN[dev]**httpport = 10080**[mail]host = $email_serverport = $email_server_portusername = $email_usernamepassword = $email_passwordfrom = $email_fromssl = $email_ssl

2.2

$ vim templates/registry/config.ymlversion: 0.1log:  level: debug  fields:    service: registrystorage:    cache:        layerinfo: inmemory    filesystem:        **rootdirectory: /var/lib/registry**    maintenance:        uploadpurging:            enabled: false    delete:        enabled: truehttp:    addr: :5000    secret: placeholder    debug:        addr: localhost:5001auth:  token:    issuer: registry-token-issuer    **realm: $ui_url:10080/service/token**    rootcertbundle: /etc/registry/root.crt    service: token-servicenotifications:  endpoints:      - name: harbor        disabled: false        url: http://ui/service/notifications        timeout: 500ms        threshold: 5        backoff: 1s

2.3

$ vim harbor.cfg## Configuration file of Harbor#The IP address or hostname to access admin UI and registry service.#DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.**hostname = 172.20.30.35**#The protocol for accessing the UI and token/notification service, by default it is http.#It can be set to https if ssl is enabled on nginx.ui_url_protocol = http#Email account settings for sending out password resetting emails.email_server = smtp.mydomain.comemail_server_port = 25email_username = [e-mail]_password = abcemail_from = admin <[e-mail]>email_ssl = false##The password of Harbor admin, change this before any production use.**harbor_admin_password = 1**##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.#Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.auth_mode = db_auth#The url for an ldap endpoint.ldap_url = ldaps://ldap.mydomain.com#The basedn template to look up a user in LDAP and verify the user's password.#For AD server, uses this template:#ldap_basedn = CN=%s,OU=Dept1,DC=mydomain,DC=comldap_basedn = uid=%s,ou=people,dc=mydomain,dc=com#The password for the root user of mysql db, change this before any production use.db_password = root123#Turn on or off the self-registration feature**self_registration = off**#Determine whether the UI should use compressed js files. #For production, set it to on. For development, set it to off.use_compressed_js = on#Maximum number of job workers in job service  max_job_workers = 3 #Determine whether the job service should verify the ssl cert when it connects to a remote registry.#Set this flag to off when the remote registry uses a self-signed or untrusted certificate.**verify_remote_cert = off**#Determine whether or not to generate certificate for the registry's token.#If the value is on, the prepare script creates new root cert and private key #for generating token to access the registry. If the value is off, a key/certificate must #be supplied for token generation.**customize_crt = off**#Information of your organization for certificatecrt_country = CNcrt_state = Statecrt_location = CNcrt_organization = organizationcrt_organizationalunit = organizational unitcrt_commonname = example.comcrt_email = [e-mail]#####

2.4

$ vim docker-compose.ymlversion: '2'services:  log:    build: ./log/    restart: always    volumes:      - /var/log/harbor/:/var/log/docker/    ports:      - 1514:514  registry:    image: library/registry:2.4.0    restart: always    volumes:      **- /var/lib/registry:/storage**      - ./config/registry/:/etc/registry/    environment:      - GODEBUG=netdns=cgo    ports:      - 5001:5001    command:      ["serve", "/etc/registry/config.yml"]    depends_on:      - log    logging:      driver: "syslog"      options:        syslog-address: "tcp://127.0.0.1:1514"        tag: "registry"  mysql:    build: ./db/    restart: always    volumes:      - /data/database:/var/lib/mysql    env_file:      - ./config/db/env    depends_on:      - log    logging:      driver: "syslog"      options:        syslog-address: "tcp://127.0.0.1:1514"        tag: "mysql"  ui:    build:      context: ../      dockerfile: Dockerfile.ui    env_file:      - ./config/ui/env    restart: always    volumes:      - ./config/ui/app.conf:/etc/ui/app.conf      - ./config/ui/private_key.pem:/etc/ui/private_key.pem    depends_on:      - log    logging:      driver: "syslog"      options:        syslog-address: "tcp://127.0.0.1:1514"        tag: "ui"  jobservice:    build:      context: ../      dockerfile: Dockerfile.job    env_file:      - ./config/jobservice/env    restart: always    volumes:      - /data/job_logs:/var/log/jobs      - ./config/jobservice/app.conf:/etc/jobservice/app.conf    depends_on:      - ui    logging:      driver: "syslog"      options:        syslog-address: "tcp://127.0.0.1:1514"        tag: "jobservice"  proxy:    image: library/nginx:1.9    restart: always    volumes:      - ./config/nginx:/etc/nginx    ports:      **- 10080:10080**      - 443:443    depends_on:      - mysql      - registry      - ui      - log    logging:      driver: "syslog"      options:        syslog-address: "tcp://127.0.0.1:1514"        tag: "proxy"

2.5

$ vim config/jobservice/app.confappname = jobservicerunmode = dev[dev]**httpport = 10080**

2.6

$ vim config/nginx/nginx.confworker_processes auto;events {  worker_connections 1024;  use epoll;  multi_accept on;}http {  tcp_nodelay on;  # this is necessary for us to be able to disable request buffering in all cases  proxy_http_version 1.1;  upstream registry {    server registry:5000;  }  upstream ui {    **server ui:10080;**  }  server {    **listen 10080;**    # disable any limits to avoid HTTP 413 for large image uploads    client_max_body_size 0;    location / {      proxy_pass http://ui/;      proxy_set_header Host $host;      proxy_set_header X-Real-IP $remote_addr;      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;      # When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.      proxy_set_header X-Forwarded-Proto $scheme;      proxy_buffering off;      proxy_request_buffering off;    }    location /v1/ {      return 404;    }    location /v2/ {      proxy_pass http://registry/v2/;      proxy_set_header Host $http_host;      proxy_set_header X-Real-IP $remote_addr;      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;      # When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.      proxy_set_header X-Forwarded-Proto $scheme;      proxy_buffering off;      proxy_request_buffering off;    }    location /service/ {      proxy_pass http://ui/service/;      proxy_set_header Host $host;      proxy_set_header X-Real-IP $remote_addr;      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;      # When setting up Harbor behind other proxy, such as an Nginx instance, remove the below line if the proxy already has similar settings.      proxy_set_header X-Forwarded-Proto $scheme;      proxy_buffering off;      proxy_request_buffering off;    }  }}

2.7

$ vim config/ui/app.confappname = registryrunmode = dev[lang]types = en-US|zh-CNnames = en-US|zh-CN[dev]**httpport = 10080**[mail]host = smtp.mydomain.comport = 25username = [e-mail] = abcfrom = admin <[e-mail]>ssl = false

2.8 

$ vim /etc/sysconfig/docker# /etc/sysconfig/docker# Modify these options if you want to change the way the docker daemon runsOPTIONS='--selinux-enabled --log-driver=journald'DOCKER_CERT_PATH=/etc/docker**DOCKER_OPTS="--insecure-registry 172.20.30.35:10080"**# If you want to add your own registry to be used for docker search and docker# pull use the ADD_REGISTRY option to list a set of registries, each prepended# with --add-registry flag. The first registry added will be the first registry# searched.#ADD_REGISTRY='--add-registry registry.access.redhat.com'# If you want to block registries from being used, uncomment the BLOCK_REGISTRY# option and give it a set of registries, each prepended with --block-registry# flag. For example adding docker.io will stop users from downloading images# from docker.io# BLOCK_REGISTRY='--block-registry'# If you have a registry secured with https but do not have proper certs# distributed, you can tell docker to not look for full authorization by# adding the registry to the INSECURE_REGISTRY line and uncommenting it.INSECURE_REGISTRY='--insecure-registry 172.20.30.35:10080'# On an SELinux system, if you remove the --selinux-enabled option, you# also need to turn on the docker_transition_unconfined boolean.# setsebool -P docker_transition_unconfined 1# Location used for temporary files, such as those created by# docker load and build operations. Default is /var/lib/docker/tmp# Can be overriden by setting the following environment variable.# DOCKER_TMPDIR=/var/tmp# Controls the /etc/cron.daily/docker-logrotate cron job status.# To disable, uncomment the line below.# LOGROTATE=false## docker-latest daemon can be used by starting the docker-latest unitfile.# To use docker-latest client, uncomment below line#DOCKERBINARY=/usr/bin/docker-latest

2.9 重启 docker-engine

$ service docker restart

或

$ systemctl restart docker.service

3、构建并启动 harbor

$ ./prepare....The configuration files are ready, please use docker-compose to start the service.$ docker-compose upStarting deploy_log_1Starting deploy_ui_1Starting deploy_mysql_1Starting deploy_registry_1Starting deploy_jobservice_1Starting deploy_proxy_1Attaching to deploy_log_1, deploy_ui_1, deploy_registry_1, deploy_mysql_1, deploy_jobservice_1, deploy_proxy_1ui_1          | 2016-07-26T13:12:15Z [INFO] Config path: /etc/ui/app.confui_1          | 2016-07-26T13:12:15Z [DEBUG] [base.go:54]: db url: mysql:3306, db user: rootui_1          | 2016-07-26T13:12:16Z [ERROR] [base.go:66]: failed to connect to db, retry after 2 seconds :dial tcp 172.18.0.5:3306: getsockopt: connection refusedregistry_1    | time="2016-07-26T13:12:15.557929687Z" level=info msg="configuring endpoint harbor (http://ui/service/notifications), timeout=500ms, headers=map[]" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0 registry_1    | time="2016-07-26T13:12:15.563872153Z" level=info msg="redis not configured" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0 registry_1    | time="2016-07-26T13:12:15.572819857Z" level=info msg="debug server listening localhost:5001" registry_1    | time="2016-07-26T13:12:15.611411032Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0 registry_1    | time="2016-07-26T13:12:15.611763465Z" level=debug msg="configured \"token\" access controller" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0 registry_1    | time="2016-07-26T13:12:15.611835323Z" level=info msg="listening on [::]:5000" go.version=go1.6.1 instance.id=d69175ba-c563-4a9a-bbcb-fc90271935e8 service=registry version=v2.4.0 mysql_1       | 2016-07-26 13:12:16 0 [Note] mysqld (mysqld 5.6.31) starting as process 1 ...mysql_1       | 2016-07-26 13:12:16 1 [Note] Plugin 'FEDERATED' is disabled.mysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Using atomics to ref count buffer pool pagesmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: The InnoDB memory heap is disabledmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtinsmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Memory barrier is not usedmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Compressed tables use zlib 1.2.8mysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Using Linux native AIOmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Using CPU crc32 instructionsmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Initializing buffer pool, size = 128.0Mmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Completed initialization of buffer poolmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Highest supported file format is Barracuda.mysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: 128 rollback segment(s) are active.mysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: Waiting for purge to startmysql_1       | 2016-07-26 13:12:16 1 [Note] InnoDB: 5.6.31 started; log sequence number 1736276mysql_1       | 2016-07-26 13:12:16 1 [Note] Server hostname (bind-address): '*'; port: 3306mysql_1       | 2016-07-26 13:12:16 1 [Note] IPv6 is available.mysql_1       | 2016-07-26 13:12:16 1 [Note]   - '::' resolves to '::';mysql_1       | 2016-07-26 13:12:16 1 [Note] Server socket created on IP: '::'.mysql_1       | 2016-07-26 13:12:16 1 [Warning] 'proxies_priv' entry '@ root@22b6d68df32a' ignored in --skip-name-resolve mode.mysql_1       | 2016-07-26 13:12:17 1 [Note] Event Scheduler: Loaded 0 eventsmysql_1       | 2016-07-26 13:12:17 1 [Note] mysqld: ready for connections.mysql_1       | Version: '5.6.31'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server (GPL)jobservice_1  | 2016-07-26T13:12:16Z [INFO] Config path: /etc/jobservice/app.confjobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:89]: config: maxJobWorkers: 3jobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:90]: config: localUIURL: http://uijobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:91]: config: localRegURL: http://registry:5000jobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:92]: config: verifyRemoteCert: offjobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:93]: config: logDir: /var/log/jobsjobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [config.go:94]: config: uiSecret: ******jobservice_1  | 2016-07-26T13:12:16Z [DEBUG] [base.go:54]: db url: mysql:3306, db user: rootjobservice_1  | 2016-07-26T13:12:16Z [ERROR] [base.go:66]: failed to connect to db, retry after 2 seconds :dial tcp 172.18.0.5:3306: getsockopt: connection refusedui_1          | 2016-07-26T13:12:18Z [INFO] User id: 1 already has its encrypted password.ui_1          | 2016/07/26 13:12:18 [asm_amd64.s:1998][I] http server Running on :<strong>10080</strong>jobservice_1  | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 0 startedjobservice_1  | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 1 startedjobservice_1  | 2016-07-26T13:12:18Z [DEBUG] [workerpool.go:123]: worker 2 startedjobservice_1  | 2016-07-26T13:12:18Z [DEBUG] [main.go:36]: Trying to resume halted jobs...jobservice_1  | 2016/07/26 13:12:18 [asm_amd64.s:1998][I] http server Running on :<strong>10080</strong>

输出类似上述信息，就说明启动成功。且 http://172.20.30.35:10080 能够访问。

如果有错误，可以从输出日志中看看是哪出错了。

查看一下其它相关信息

$ docker ps -aCONTAINER ID        IMAGE                    COMMAND                  CREATED             STATUS              PORTS                                                    NAMES7f893cfcead0        library/nginx:1.9        "nginx -g 'daemon off"   About an hour ago   Up About a minute   0.0.0.0:443->443/tcp, 80/tcp, 0.0.0.0:10080->10080/tcp   deploy_proxy_1e70967b2b72f        deploy_jobservice        "/go/bin/harbor_jobse"   About an hour ago   Up About a minute                                                            deploy_jobservice_1975e74c43e47        deploy_mysql             "docker-entrypoint.sh"   About an hour ago   Up About a minute   3306/tcp                                                 deploy_mysql_1e4b364187a43        library/registry:2.4.0   "/bin/registry serve "   About an hour ago   Up About a minute   5000/tcp, 0.0.0.0:5001->5001/tcp                         deploy_registry_1e91872374e25        deploy_ui                "/go/bin/harbor_ui"      About an hour ago   Up About a minute   80/tcp                                                   deploy_ui_104d2ddf3cd63        deploy_log               "/bin/sh -c 'cron && "   About an hour ago   Up About a minute   0.0.0.0:1514->514/tcp                                    deploy_log_1

$ netstat -nlpt | grep docker-proxyActive Internet connections (only servers)Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name           tcp6       0      0 :::5001                 :::*                    LISTEN      13272/docker-proxy  tcp6       0      0 :::1514                 :::*                    LISTEN      13177/docker-proxy  tcp6       0      0 :::443                  :::*                    LISTEN      13442/docker-proxy  tcp6       0      0 :::10080                :::*                    LISTEN      13435/docker-proxy

到这里，源码搭建就算完成了。

预构建安装

预构建安装 跟 源码 安装几乎一样。

预构建安装 支持 无外网连接安装，其实就是先在能连外网的机器上安装一遍，再用 docker save 将镜像导出来，再把导出来的镜像用 scp 或其它工具传给目标主机，再在目标主机上用 docker load 导进去。之后步骤就是上面讲的改配置等等。

使用

1、在你的机器上修改文件 /etc/sysconfig/docker ，添加（或修改）字段：

OPTIONS='--selinux-enabled --log-driver=journald'DOCKER_CERT_PATH=/etc/dockerADD_REGISTRY='--add-registry 172.20.30.35:10080'DOCKER_OPTS="--insecure-registry 172.20.30.35:10080"INSECURE_REGISTRY='--insecure-registry 172.20.30.35:10080'

2、重启 docker-engine

$ service docker restart

或

$ systemctl restart docker.service

3、登录

$ docker login 172.20.30.35:10080

4、push 镜像到仓库
4.1, 先tag

$ docker tag your-docker-image 172.20.30.35:10080/your-repository-name/your-docker-image

注意，your-repository-name 不能少，否则会报错：权限验证失败
4.2, push 到 172.20.30.35:10080 上去

$ docker push 172.20.30.35:10080/your-repository-name/your-docker-image

5、从仓库中 pull 镜像

$ docker pull your-repository-name/your-docker-image

或者

$ docker pull 172.20.30.35:10080/your-repository-name/your-docker-image

参考：http://1.chaoxu.sinaapp.com/archives/3969