Bluemix平台一步一步搭建ZooKeeper + Mesos + Marathon平台管理Docker集群之二

接上一篇文章：http://blog.csdn.net/rapheler/article/details/52229208

咱们来看一下Marathon跑一个Docker任务

Marathon的界面如下：

利用这个UI界面可以很容易的创建应用，并且可以以json的方式编辑，只要右上角这个JSON Mode被选中即可：

比如我们创建一个python的应用程序，来看看后来Docker是否正常运行：

有了这个平台创建基于Docker的应用非常容易，而且扩展起来也是秒级的，比如我们创建了一个Nginx的应用，然后需要扩展到5个，只需要点击Scale Application，输入5，后台就自动扩展了5个Nginx的应用。

然后就是故障自动转移，加入我们去后台终止一个正在运行的容器，Marathon会自动检测到正在运行的任务与设置的不一样，而立马新创建一个应用。

初了用web ui的方式来创建任务，也可以使用API的形式：

比如我这里有个2048游戏的json格式的文件，使用的image为nginx:latest，将本地的/home/ibmcloud/2048-maser/ 映射到容器的nginx根目录，采用BRIDGE网络，expose容器的80端口，并对改容器进行http的健康检查：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

$ cat nginx-bridge-2048game.json 

{

  "id": "/nginx",

  "cpus": 1,

  "mem": 128,

  "disk": 0,

  "instances": 1,

  "container": {

    "type": "DOCKER",

    "volumes": [

      {

        "containerPath": "/usr/share/nginx/html/",

        "hostPath": "/home/ibmcloud/2048-master/",

        "mode": "RO"

      }

    ],

    "docker": {

      "image": "nginx",

      "network": "BRIDGE",

      "portMappings": [

        {

          "containerPort": 80,

          "hostPort": 0,

          "protocol": "tcp",

          "servicePort": 80,

          "labels": {}

        }

      ],

      "privileged": false,

      "parameters": [],

      "forcePullImage": false

    }

  },

  "healthChecks": [

    {

      "path": "/",

      "protocol": "HTTP",

      "portIndex": 0,

      "gracePeriodSeconds": 120,

      "intervalSeconds": 30,

      "timeoutSeconds": 5,

      "maxConsecutiveFailures": 3,

      "ignoreHttp1xx": false

    }

  ],

  "portDefinitions": [

    {

      "port": 10003,

      "protocol": "tcp",

      "labels": {}

    }

  ]

}

在Marathon的Leader机器上运行

1

2

3

4

5

6

7

8

9

10

11

12

# curl -u username:password -i -H 'Content-Type: application/json' [e-mail] 192.168.0.33:8080/v2/apps 

HTTP/1.1 201 Created

Date: Tue, 07 Jun 2016 07:59:17 GMT

X-Marathon-Leader: http://192.168.0.33:8080

Cache-Control: no-cache, no-store, must-revalidate

Pragma: no-cache

Expires: 0

Location: http://192.168.0.33:8080/v2/apps/nginx

Content-Type: application/json; qs=2

Transfer-Encoding: chunked

Server: Jetty(9.3.z-SNAPSHOT)

{"id":"/nginx","cmd":null,"args":null,"user":null,"env":{},"instances":1,"cpus":1,"mem":128,"disk":0,"executor":"","constraints":[],"uris":[],"fetch":[],"storeUrls":[],"ports":[80],"portDefinitions":[{"port":80,"protocol":"tcp","labels":{}}],"requirePorts":false,"backoffSeconds":1,"backoffFactor":1.15,"maxLaunchDelaySeconds":3600,"container":{"type":"DOCKER","volumes":[{"containerPath":"/usr/share/nginx/html/","hostPath":"/home/ibmcloud/2048-master/","mode":"RO"}],"docker":{"image":"nginx","network":"BRIDGE","portMappings":[{"containerPort":80,"hostPort":0,"servicePort":80,"protocol":"tcp","labels":{}}],"privileged":false,"parameters":[],"forcePullImage":false}},"healthChecks":[{"path":"/","protocol":"HTTP","portIndex":0,"gracePeriodSeconds":120,"intervalSeconds":30,"timeoutSeconds":5,"maxConsecutiveFailures":3,"ignoreHttp1xx":false}],"readinessChecks":[],"dependencies":[],"upgradeStrategy":{"minimumHealthCapacity":1,"maximumOverCapacity":1},"labels":{},"acceptedResourceRoles":null,"ipAddress":null,"version":"2016-06-07T07:59:18.003Z","residency":null,"tasksStaged":0,"tasksRunning":0,"tasksHealthy":0,"tasksUnhealthy":0,"deployments":[{"id":"c30b9f6e-786c-45bd-9e7e-cc6cc001d91d"}],"tasks":[]}#

 为了方便，可以将这个命令写入到脚本里运行：

1

2

3

$ cat runapp.sh 

#!/bin/bash

curl -i -H 'Content-Type: application/json' -d@"$1" 192.168.0.33:8080/v2/apps

可以看到我们的2048游戏已经可以玩了：

最后强调一下安全问题，因为我把Marathon暴漏到公网了，而且没加认证，导致有坏蛋扫描到我的机器，并且创建了一个Metasploit的任务对我的服务器进行渗透，好在我的安全性做的还可以，凶手没有得逞，参考这篇文章：http://shanker.blog.51cto.com/1189689/1785797

然后marathon开启认证的方式是这样运行的：

1

marathon --http_credentials "username:password"

注意问题：

1. 需要添加MESOS_QUORUM到/etc/default/mesos-master， 即使/etc/mesos-master/quorum 有这个，也必须要做这一步（来自stackoverflow的解决方案）

# cat /etc/default/mesos-master

PORT=5050

ZK=`cat /etc/mesos/zk`

MESOS_QUORUM=`cat /etc/mesos-master/quorum`

2.  slave start fails, need to delete the slave.info file /tmp/mesos/meta/slaves/latest/slave.info

3.当遇到主节点不是当前节点，浏览器自动转发的时候，用到的转发的文件是/etc/mesos-master/hostname，是每台机器的hostname只写自己的ip地址，而不是将所有master节点ip都写上，
然后marathon 的hostname同master的hostname，cp 过去即可。

4.遇到这个问题是因为你的slave没有禁用mesos-master，或者开启了mesos-master进程

Failed to create ZooKeeper, zookeeper_init: No such file or directory

5.遇到以下问题需要查看/etc/mesos-master/quorum 是否都设置的数量，而且/etc/default/mesos-master配置文件也有QUORUM的设置，请参考问题1

Replica in VOTING status received a broadcasted recover request from 192.168.0.33:37798

6. 查看mesos 网页的的LOG ，出现以下内容，才是正常的

I0524 07:53:33.242832 27506 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'
I0524 07:53:41.834223 27507 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:53:44.241183 27509 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'
I0524 07:53:52.034374 27502 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:53:55.241883 27504 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'
I0524 07:54:02.236343 27505 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:54:06.243017 27508 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'
I0524 07:54:13.141284 27507 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:54:22.238883 27509 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'
I0524 07:54:23.345165 27509 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:54:33.545162 27509 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'
I0524 07:54:34.240898 27503 http.cpp:312] HTTP GET for /master/state from 124.193.167.1:18148 with User-Agent='Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.75 Safari/537.36'

I0524 07:54:43.755383 27504 http.cpp:312] HTTP GET for /master/state from 192.168.0.33:37798 with User-Agent='Mozilla/5.0 (Windows NT 6.1; rv:46.0) Gecko/20100101 Firefox/46.0'

8.要支持Docker任务，一定要添加这两个文件到salve下。

/etc/mesos-slave/{containerizers,executor_registration_timeout }

参考文档：

https://open.mesosphere.com/getting-started/install/

https://www.youtube.com/watch?v=hZNGST2vIds&feature=youtu.be

https://www.youtube.com/watch?v=EgYyf3bSb8Q

https://www.youtube.com/watch?v=_uw1ISM_uRU