bigchaindb集群部署（5节点）

在安装bigchainDB集群之前，我们需要了解bigchainDB的一些相关术语：

• bigchainDB节点：是一台或一群相近的节点的集合。bigchainDB节点上运行了mongoDB server、bigchainDB server及一些相关的软件
• bigchainDB集群：一群互联的bigchainDB节点组成了bigchainDB集群。集群中的节点运行相同的软件。一个集群可能需要有其他的额外的节点来完成集群监控
• bigchainDB联盟（Consortium）：一个集群中管理节点的组织或个人属于bigchainDB联盟。当集群由一个公司管理运行，则联盟为该公司

bigchainDB的节点按照使用目的可以分为dev/test节点（开发与测试）、bare-bones节点（部署在云上）或者production节点（拥有更多的组件）。一个production节点必须包括BigchainDB Server、MongoDB Server 3.4+ (mongod)以及Scalable storage for MongoDB。这些关键组件的关系如下图所示。其中BigChainDB server必须能够与所有mongod实例相连，所有的mongod也相互连接。

安装环境

hostname ip os node-admin 10.0.0.70 ubuntu 14.04 desktop node1 10.0.0.71 ubuntu 14.04 server node2 10.0.0.72 ubuntu 14.04 server node3 10.0.0.73 ubuntu 14.04 server node4 10.0.0.74 ubuntu 14.04 server node5 10.0.0.75 ubuntu 14.04 server

其中node-admin用来使用ansible来执行远程命令（为了部署便利），node1-node5为bigchain DB节点。所有机器用户均为root

为了使用ansible，我们首先在所有节点上安装openssh-server，并允许root用户ssh。具体方式为修改/etc/ssh/sshd_config，修改为PermitRootLogin yes，然后重启ssh服务。

安装与配置ansible

接下来在node-admin上安装与配置ansible。

apt-get update && apt-get upgradeapt-get install ansible

修改/etc/ansible/hosts:

root@clean:~# grep -Ev  "^$|#" /etc/ansible/hosts[bigchain]10.0.0.71 ansible_ssh_pass=12310.0.0.72 ansible_ssh_pass=12310.0.0.73 ansible_ssh_pass=12310.0.0.74 ansible_ssh_pass=12310.0.0.75 ansible_ssh_pass=123root@clean:~#

设置为首次链接不需要key认证

sed -i "s/^#\(host_key_checking\).*/\1 = False/g" /etc/ansible/ansible.cfg

测试：ansible bigchain -m ping

安装bigchainDB

以下所有命令默认在node-admin上执行

• 同步时钟

# 测试是否能ping通cn.pool.ntp.organsible bigchain -m command -a "ping cn.pool.ntp.org -c 4"# 同步ansible bigchain -m command -a "ntpdate cn.pool.ntp.org"

• 安装mongoDB

编辑sources.list

echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

备用 清华的源

echo "deb [ arch=amd64 ] https://mirrors.tuna.tsinghua.edu.cn/mongodb/apt/ubuntu trusty/mongodb-org/3.4 multiverse" > mongodb-org-3.4.list

编辑mongod.yml，内容如下：

---- hosts: bigchain  remote_user: root  # invoke setup module to gather facts before executing tasks  gather_facts: true  tasks:  - name: debug    debug: msg="myhostname={{ansible_hostname}}"  - name: apt-key    command: sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6  - name: sources.list    template:       src: mongodb-org-3.4.list      dest: /etc/apt/sources.list.d/mongodb-org-3.4.list      owner: root      group: root      mode: 0644  - name: update    command: apt-get update -y  - name: install packages    apt:      name: "{{item}}"      force: yes    with_items:      - mongodb-org      - g++       - python3-dev      - libffi-dev      - python3-pip  - name: setuptools    command: pip3 install --upgrade pip setuptools  - name: db    command: mkdir -p /data/db

运行yml(耗时很长，因为需要完成apt-get update与安装许多包)

ansible-playbook mongod.yml

• 启动mongod

# 启动mongoansible bigchain -m command -a "mongod --replSet bigchain --fork --logpath /var/log/mongodb/mongod.log"

若出现address已经被绑定这种错误的话，可以在错误节点上执行

kill -9 $(ps -ef | grep mongod | awk 'NR>1{print p}{p=$2}')# kill全部ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep mongod | awk 'NR>1{print p}{p=\$2}')"

若mongod出现错误：

2017-06-21T09:56:34.552+0800 W -        [initandlisten] Detected unclean shutdown - /var/lib/mongodb/mongod.lock is not empty.2017-06-21T09:56:34.568+0800 E NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted2017-06-21T09:56:34.568+0800 I -        [initandlisten] Fatal Assertion 28578 at src/mongo/util/net/listen.cpp 195

可以在错误节点上执行：

rm /data/db/mongod.lock  mongod --repairansible bigchain -m raw -a "rm /data/db/mongd.lock"ansible bigchain -m raw -a "mongod --repair"

• 创建副本集

在node1上执行

mongo    config = {_id: 'bigchain', members: [{    "_id": 0,    "host":"10.0.0.71:27017"    }]    }    rs.initiate(config);    rs.add("10.0.0.72:27017")    rs.add("10.0.0.73:27017")    rs.add("10.0.0.74:27017")    rs.add("10.0.0.75:27017")

• 安装bigchainDB

ansible bigchain -m command -a "pip3 install --upgrade bigchaindb"ansible bigchain -m command -a "bigchaindb -y configure mongodb"

• 修改bigchainDB配置

允许接收所有地址信息

ansible bigchain -m raw -a "sed -i 's/\(\"bind\": \"\)localhost:9984\"/\10.0.0.0:9984\"/g' /root/.bigchaindb"

修改replSet名称为bigchain（之前mongod的replset名称）：

ansible bigchain -m raw -a "sed -i 's/\(\"replicaset\": \"\).*/\1bigchain\",/g' /root/.bigchaindb"

还需要修改keyring选项，使之存储除本节点之外的所有其他的节点的keyring的公钥。先来获取所有节点的keyring的公钥

ansible bigchain -m raw -a "cat .bigchaindb | grep public | awk -F\\\" '{print \$4}'"

输出如下：

10.0.0.71 | success | rc=0 >>7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ10.0.0.72 | success | rc=0 >>GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv10.0.0.74 | success | rc=0 >>Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv10.0.0.73 | success | rc=0 >>8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv10.0.0.75 | success | rc=0 >>GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH

我们将这些信息写入一个脚本(conf.py)里

import sysimport jsonkeyring = {    "10.0.0.71": "7772APkwHENC8j3tDaUK2WJYPF3AMrTkVgR7sW1y3bkZ",    "10.0.0.72": "GRTkTmFuYETDaXAftSZW1SdCMMwaYs6p6yhAn5C4QBZv",    "10.0.0.74": "Eok1FnDbKpak9t6SpJVpFsMqkvNiVGsys6BP8UbSiCTv",    "10.0.0.73": "8bXEbEJVCDNhptYyAJ5WWHCngiie6VuwTKF5NmZ4Fazv",    "10.0.0.75": "GH3uAPwi1MzXsxy4PJdj4p5m55nXuLAakNtpFNJw7cqH"}rets = []for key, value in keyring.items():    if key != sys.argv[1]:        rets.append(value)conf = json.load(open("/root/.bigchaindb"))conf['keyring'] = retsjson.dump(conf, open("/root/.bigchaindb", "w"), indent=2)

编辑bigchain.yml，用来分发该脚本，并执行该脚本

---- hosts: bigchain  remote_user: root  # invoke setup module to gather facts before executing tasks  gather_facts: true  tasks:  - name: debug    debug: msg="my ip of eth0 is {{ansible_eth0.ipv4.address}}"  - name: copy file    template:      src: conf.py      dest: /root/conf.py      owner: root      group: root      mode: 0644  - name: modify configuration    command: python conf.py {{ansible_eth0.ipv4.address}}

执行：

ansible-playbook bigchain.yml

执行成功后，可以看到node1-node5的.bigchaindb里keyring均成功写入。

• 启动bigchainDB

任选一节点（如node1上）执行

bigchaindb init

然后启动所有节点的bigchaindb

bigchaindb start# 或者后台启动nohup bigchaindb start > /dev/null 2>&1 &# 启动全部ansible bigchain -m shell -a "nohup bigchaindb start > /dev/null 2>&1 &"# kill全部ansible bigchain -m raw -a "kill -9 \$(ps -ef | grep bigchaindb | awk 'NR>1{print p}{p=\$2}')"

使用bigchaindb

首先得安装bigchaindb的python driver

ansible bigchain -m command -a "apt-get install git -y"ansible bigchain -m command -a "apt-get install libssl-dev -y"ansible bigchain -m command -a "pip3 install --process-dependency-links git+https://github.com/bigchaindb/bigchaindb-driver.git"

官网上的测试用例：https://docs.bigchaindb.com/projects/py-driver/en/latest/usage.html

# coding=utf-8# author: Wu Luoimport bigchaindb_driverfrom bigchaindb_driver import BigchainDBfrom bigchaindb_driver.crypto import generate_keypairfrom time import sleepimport jsonfrom sys import exitimport sysclass BigChainAPI(object):    def __init__(self, host, port=9984, conf='/root/.bigchaindb', timeout=60):        '''        :param host: cluster host of bigchaindb        :param port: port of bigchaindb        :param conf: the configuration file of bigchaindb        :param timeout: waiting vote for `timeout` seconds        :return:        '''        self.host = host        self.port = int(port)        self.bdb = BigchainDB("http://%s:%d" % (self.host, self.port))        self.conf = conf        self.timeout = int(timeout)        self.loadUserKey()    def loadUserKey(self):        '''        load user key from configuration files, as the valid keypairs are         designated in our deployment        :return:        '''        configuration = json.load(open(self.conf, "r"))        self.user = configuration['keypair']    def waitVote(self, txid):        '''        wait the transaction to be voted as valid        :param txid: the transaction id        :return: True/False        '''        print("txid is ", txid)        trials = 0        while trials < self.timeout:            try:                if self.bdb.transactions.status(txid).get('status') == 'valid':                    print('Tx valid in:', trials, 'secs')                    break                elif self.bdb.transactions.status(txid).get('status') == 'invalid':                    print('Tx invalid in:', trials, 'secs')                    print(self.bdb.transactions.status(txid))                    break                else:                    trials += 1                    print("trials " + str(trials))                    sleep(1)            except bigchaindb_driver.exceptions.NotFoundError:                trials += 1                sleep(1)        if trials == self.timeout:            print('Tx is still being processed... Bye!')            return False        return True    def writeTransaction(self, data, metadata=None):        '''        write data into bigchaindb        :param data: the data to write. it should be a dict        :param metadata: the metadata to write. it should be a dict        :return: the transaction id        '''        _asset = {            'data': data        }        prepared_creation_tx = self.bdb.transactions.prepare(            operation='CREATE',            signers=self.user['public'],            asset=_asset,            metadata=metadata        )        fulfilled_creation_tx = self.bdb.transactions.fulfill(            prepared_creation_tx,            private_keys=self.user['private']        )        sent_creation_tx = self.bdb.transactions.send(fulfilled_creation_tx)        txid = fulfilled_creation_tx['id']        if not self.waitVote(txid):            return False        return txid    def transferAsset(self, asset_id, publicKeyOfTargetUser):        '''        transfer an asset to another user        :param asset_id: the asset to transfer        :param publicKeyOfTargetUser: the public key of the target user         It should be generated by generate_keypair()        :return: the transaction id        '''        transfer_asset = {            'id': asset_id        }        # query the transaction which contains the asset        data = self.bdb.transactions.retrieve(asset_id)        output_index = 0        output = data['outputs'][output_index]        transfer_input = {            'fulfillment': output['condition']['details'],            'fulfills': {                'output': output_index,                # in other version of BigchainDB, the key should be 'txid'                'transaction_id': data['id']            },            'owners_before': output['public_keys']        }        prepared_transfer_tx = self.bdb.transactions.prepare(            operation='TRANSFER',            asset=transfer_asset,            inputs=transfer_input,            recipients=publicKeyOfTargetUser,        )        fulfilled_transfer_tx = self.bdb.transactions.fulfill(            prepared_transfer_tx,            private_keys=self.user['private'],        )        sent_transfer_tx = self.bdb.transactions.send(fulfilled_transfer_tx)        txid = fulfilled_transfer_tx['id']        if not self.waitVote(txid):            return False        return txid    def queryTransaction(self, txid):        '''        retrieve a transaction        :param txid: the txid of the transaction to query        :return: the data of the retrieved transaction        '''        data = self.bdb.transactions.retrieve(txid)        return data['asset']def test():    bigchainAPI = BigChainAPI(host='10.0.0.71', port=9984, timeout=300)    # write an transaction    data = {        'author': 'Wu Luo'    }    txid = bigchainAPI.writeTransaction(data)    if txid == False:        print(">>> waiting for vote")        return False    # query this transaction    query_data = bigchainAPI.queryTransaction(txid)    print(">>> txid is " + txid)    print(">>> data retrieved from bigchainDB")    print(query_data)    # transfer    user = generate_keypair()    newid = bigchainAPI.transferAsset(txid, user.public_key)    print(txid, newid)    transfer_data = bigchainAPI.queryTransaction(newid)    print(transfer_data)test()

运行后

root@node1:~# python3 bigchainAPI.pytxid is  6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5trials 1trials 2Tx valid in: 2 secs>>> txid is 6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5>>> data retrieved from bigchainDB{'data': {'author': 'Wu Luo'}}txid is  c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8trials 1trials 2Tx valid in: 2 secs6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5 c6a7b301d485243a7404fd048d0cec3912410f58ccd20701bc8937d7587516a8{'id': '6a82600a7cd8683f8e83cc6cfb765b219a8543404da2a9ad5b56114b9f1a8dc5'}

部署成功