mongodb 在单机上搭建分片集群 详细实战过程
来源:互联网 发布:振华重工 知乎 编辑:程序博客网 时间:2024/05/22 20:18
<->本地搭建
一般线上mongodb服务器都好多台,可以自由搭建mongodb分片集群的,但是不能随意测试,如果想自己随意测试新功能怎么办呢?可以在本地搭建,在一台服务器mongodb1(hch_test_dbm1_121_62)上大家mongodb分片集群,下面分片是示意图(https://docs.mongodb.com/manual/_images/sharded-cluster-production-architecture.png):
OK,接下来就准备开始搭建mongodb分片集群,从安装mongodb开始,然后逐渐部署config servers、mongos servers、shard servers一步步来开始准备。
1,下载安装:
https://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.2.3.tgz/download,我这里下载的是稳定版本:mongodb-linux-x86_64-3.0.3.tgz
下载后解压缩
tar -xvf mongodb-linux-x86_64-3.0.3.tgz -C/usr/local/
mkdir /data
mkdir/usr/local/mongodb-linux-x86_64-3.0.3/log/
mkdir /data/mongodb
chown -R mongodb.mongodb /data/mongodb
chown -R mongodb.mongodb/usr/local/mongodb-linux-x86_64-3.0.3
chown -R mongodb:mongodb/usr/local/mongodb-linux-x86_64-3.0.3 -R
chown -R mongodb:mongodb /data/mongodb -R
2、在mongodb1上准备好3个config servers
创建数据目录日志目录:
mkdir /data/mongodb/config20001 -p
mkdir /data/mongodb/config20002 -p
mkdir /data/mongodb/config20003 -p
mkdir /data/mongodb/logs/ -p
启动config servers 进程:
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20001 --port 20001 --logpath/data/mongodb/logs/configsvr_20001.log --logappend --fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20002 --port 20002 --logpath/data/mongodb/logs/configsvr_20002.log --logappend --fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--configsvr --dbpath /data/mongodb/config20003 --port 20003 --logpath/data/mongodb/logs/configsvr_20003.log --logappend --fork
3、启动mongos路由服务
在mongodb1上启动mongos服务器(可以启动一个也可以启动多个):
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongos--configdb mongodb1:20001,mongodb1:20002,mongodb1:20003 --port 40000 --chunkSize 64 --logpath/data/mongodb/logs/mongos40004.log --logappend --fork
4、开始准备shard servers
# 第一个shard1分片副本集
# 创建目录
mkdir /data/mongodb/shard37017
mkdir /data/mongodb/shard37027
mkdir /data/mongodb/shard37037
# 开始启动shard servers进程
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37017 --dbpath /data/mongodb/shard37017--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37017.log --logappend--fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37027 --dbpath /data/mongodb/shard37027--oplogSize 2048 --logpath /data/mongodb/logs/shard_m1s1_37027.log --logappend--fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard1 --port 37037 --dbpath /data/mongodb/shard37037--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37037.log --logappend--fork
# 第二个shard2分片副本集
# 创建目录
mkdir /data/mongodb/shard37018
mkdir /data/mongodb/shard37028
mkdir /data/mongodb/shard37038
# 启动shard severs进程
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37018 --dbpath /data/mongodb/shard37018--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37018.log --logappend--fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37028 --dbpath /data/mongodb/shard37028--oplogSize 2048 --logpath /data/mongodb/logs/shard_m1s1_37028.log --logappend--fork
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod--shardsvr --replSet shard2 --port 37038 --dbpath /data/mongodb/shard37038--oplogSize 2048 --logpath /data/mongodb/logs/shard_m11_37038.log --logappend--fork
5、开始设置分片副本集shard1
# 设置第一个分片副本集,必须使用admin数据库,登录命令如下
/usr/local/mongodb-linux-x86_64-3.0.3/bin/mongomongodb1:37017/admin
#定义副本集命令
> config = { _id:"shard1",members:[
{_id:0,host:"mongodb1:37017",priority:1},
{_id:1,host:"mongodb1:37027",priority:2},
{_id:2,host:"mongodb1:37037",arbiterOnly:true}
]
};
# 初始化副本集命令
> rs.initiate(config);
# 执行过程如下:
[mongodb@dnadb_1 mongodb]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017/admin
MongoDB shell version: 3.0.3
connecting to: localhost:27017/admin
Server has startup warnings:
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-04-07T23:41:07.703+0800 I CONTROL [initandlisten]
> config = { _id:"shard1", members:[
... {_id:0,host:"mongodb1:37017",priority:1},
... {_id:1,host:"mongodb1:37027",priority:2},
... {_id:2,host:"mongodb1:37037",arbiterOnly:true}
... ]
... };
{
"_id" : "shard1",
"members" : [
{
"_id" : 0,
"host" : "mongodb1:37017",
"priority" : 1
},
{
"_id" : 1,
"host" : "mongodb1:37027",
"priority" : 2
},
{
"_id" : 2,
"host" : "mongodb1:37037",
"arbiterOnly" : true
}
]
}
> rs.initiate(config);
{ "ok" : 1 }
shard1:OTHER>
6、开始设置分片副本集shard2
# 开始设置第二个分片副本集
> config = { _id:"shard2",members:[
{_id:0,host:"mongodb1:37018",priority:1},
{_id:1,host:"mongodb1:37028",priority:2},
{_id:2,host:"mongodb1:37038",arbiterOnly:true}
]
};
# 初始化副本集
> rs.initiate(config);
执行过程如下:
[mongodb@dnadb_1 mongodb]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27018/admin
MongoDB shell version: 3.0.3
connecting to: localhost:27018/admin
Server has startup warnings:
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-04-07T23:42:18.944+0800 I CONTROL [initandlisten]
> config = { _id:"shard2", members:[
... {_id:0,host:"mongodb1:37018",priority:1},
... {_id:1,host:"mongodb1:37028",priority:2},
... {_id:2,host:"mongodb1:37038",arbiterOnly:true}
... ]
... };
{
"_id" : "shard2",
"members" : [
{
"_id" : 0,
"host" : "mongodb1:37018",
"priority" : 1
},
{
"_id" : 1,
"host" : "mongodb1:37028",
"priority" : 2
},
{
"_id" : 2,
"host" : "mongodb1:37038",
"arbiterOnly" : true
}
]
}
> rs.initiate(config);
{ "ok" : 1 }
shard2:OTHER>
7、检查下副本状态:
[mongodb@hch_test_dbm1_121_62 ~]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongomongodb1:37028/admin
MongoDB shell version: 3.0.3
connecting to: mongodb1:37028/admin
Server has startup warnings:
2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten]
2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** WARNING: You are runningon a NUMA machine.
2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** We suggest launching mongod like thisto avoid performance problems:
2016-07-09T19:08:58.250+0800 I CONTROL [initandlisten] ** numactl --interleave=all mongod[other options]
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** WARNING:/sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** WARNING:/sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2016-07-09T19:08:58.251+0800 I CONTROL [initandlisten]
shard2:PRIMARY> rs.status();
{
"set": "shard2",
"date": ISODate("2016-07-09T11:35:36.680Z"),
"myState": 1,
"members": [
{
"_id": 0,
"name": "mongodb1:37018",
"health": 1,
"state": 2,
"stateStr": "SECONDARY",
"uptime": 62,
"optime": Timestamp(1468064073, 1),
"optimeDate": ISODate("2016-07-09T11:34:33Z"),
"lastHeartbeat": ISODate("2016-07-09T11:35:35.736Z"),
"lastHeartbeatRecv": ISODate("2016-07-09T11:35:35.739Z"),
"pingMs": 0,
"configVersion": 1
},
{
"_id": 1,
"name": "mongodb1:37028",
"health": 1,
"state": 1,
"stateStr": "PRIMARY",
"uptime": 1598,
"optime": Timestamp(1468064073, 1),
"optimeDate": ISODate("2016-07-09T11:34:33Z"),
"electionTime": Timestamp(1468064110, 1),
"electionDate": ISODate("2016-07-09T11:35:10Z"),
"configVersion": 1,
"self": true
},
{
"_id": 2,
"name": "mongodb1:37038",
"health": 1,
"state": 7,
"stateStr": "ARBITER",
"uptime": 62,
"lastHeartbeat": ISODate("2016-07-09T11:35:35.747Z"),
"lastHeartbeatRecv": ISODate("2016-07-09T11:35:35.745Z"),
"pingMs": 0,
"configVersion": 1
}
],
"ok": 1
}
shard2:PRIMARY>
8、设置自动分片生效
目前搭建了mongodb配置服务器、路由服务器,各个分片服务器,不过应用程序连接到 mongos 路由服务器并不能使用分片机制,还需要在程序里设置分片配置,让分片生效。
# 串联路由服务器与分配副本集命令
db.runCommand( { addshard :"shard1/mongodb1:37017,mongodb1:37027,mongodb1:37037"});
db.runCommand( { addshard :"shard2/mongodb1:37018,mongodb1:37028,mongodb1:37038"});
# 执行过程
[mongodb@db_m1_slave_1 logs]$ /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongolocalhost:40000/admin
MongoDB shell version: 3.0.3
connecting to: localhost:30000/admin
mongos> db.runCommand( { addshard :"shard1/mongodb1:37017,mongodb1:37027,mongodb1:37037"});
{ "shardAdded" :"shard1", "ok" : 1 }
mongos> db.runCommand( { addshard :"shard2/mongodb1:37018,mongodb1:37028,mongodb1:37038"});
{ "shardAdded" :"shard2", "ok" : 1 }
mongos>
9、查看分片服务器配置
db.runCommand({listshards : 1 });
命令输出结果
mongos> db.runCommand({listshards : 1});
{
"shards": [
{
"_id": "shard1",
"host": "shard1/mongodb1:37017,mongodb1:37027"
},
{
"_id": "shard2",
"host": "shard2/mongodb1:37018,mongodb1:37028"
}
],
"ok": 1
}
mongos>
PS:因为37037、37038是每个分片副本集的仲裁节点,所以在上面结果没有列出来。
查看均衡器,默认是打开的:
mongos> sh.getBalancerState();
true
mongos>
10、如何让录入的数据自动分片
目前配置服务、路由服务、分片服务、副本集服务都已经串联起来了,但我们的目的是希望插入数据,数据能够自动分片,那么还需要做如下工作:
连接在mongos上,准备让指定的数据库、指定的集合分片生效。
# 指定分片生效
db.runCommand( { enablesharding:"app"});
db.runCommand( { enablesharding:"im"});
db.runCommand( { enablesharding:"parking"});
db.runCommand( { enablesharding:"pv"});
db.runCommand( { enablesharding:"report"});
db.runCommand( { enablesharding:"screen"});
db.runCommand( { enablesharding:"search"});
db.runCommand( { enablesharding :"traffice"});
db.runCommand( { enablesharding:"wifi"});
执行过程如下:
mongos> db.runCommand( { enablesharding:"app"});
});
db.runCommand( { enablesharding:"search"});
db.runCommand( { enablesharding:"traffice"});
db.runCommand( { enablesharding:"wifi"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"im"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"parking"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"pv"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"report"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"screen"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"search"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"traffice"});
{ "ok" : 1 }
mongos> db.runCommand( { enablesharding:"wifi"});
{ "ok" : 1 }
mongos>
mongos>
# 指定数据库里需要分片的集合和片键
db.runCommand( { shardcollection :"uba.table1",key : {id: 1} } )
我们设置uba的 table1 表需要分片,根据 id 自动分片到 shard1 ,shard2,shard3 上面去。要这样设置是因为不是所有 mongodb 的数据库和表都需要分片!
--1 app
mongos> use bg;
switched to db bg
mongos> db.card.save({x:1});
WriteResult({ "nInserted" : 1 })
mongos>
sh.enableBalancing("bg");
db.runCommand({enablesharding:"app"});
db.runCommand( { shardcollection :"app.download",key : {_id: 1} } );
执行过程如下:
mongos>db.runCommand({enablesharding:"app"});
{"ok" : 0, "errmsg" : "already enabled" }
mongos>use admin
switchedto db admin
mongos>db.runCommand({enablesharding:"app"});
{"ok" : 0, "errmsg" : "already enabled" }
mongos>db.runCommand( { shardcollection : "app.download",key : {_id: 1} } );
{"collectionsharded" : "app.download", "ok" : 1 }
mongos>
11、配置片键
需要进入admin数据库里面去执行配置命令:
use admin
db.runCommand({"shardcollection":"201.customer_user_mst",key:{"name":1}})
db.runCommand({"shardcollection":"201.user_attr_mst",key:{"name":1}})
db.runCommand({"shardcollection":"pv.pv1",key:{"_id":1}})
db.pv1.insert({ "_id" :ObjectId("4b8ed00a1d42d47b3afa3c41"), "x" : 999,"note" : "in db2" });
- mongodb 在单机上搭建分片集群 详细实战过程
- MongoDB分片实战(一):集群搭建
- MongoDB分片实战(一):集群搭建
- MongoDB分片实战(一):集群搭建
- MongoDB分片实战(一):集群搭建
- MongoDB分片实战(一):集群搭建
- MongoDB分片实战(一):集群搭建
- MongoDB分片集群实战
- MongoDB 分片集群搭建
- mongodb集群搭建-分片
- MongoDB分片集群搭建
- mongodb 集群分片搭建
- MongoDB实战-分片集群实战
- MongoDB分片实战(一):集群搭建 环境准备
- 快速搭建MongoDB分片集群
- mongodb分片集群的搭建
- 在Docker上部署mongodb分片副本集群。
- 在服务器上设置mongodb分片集群开机自启动
- 690C2Brain Network (medium)
- 提高项目30.6-删除字符串前面的空格
- [2D图形处理]简易Alpha混合算法
- Laravel请求和输入
- 关于timeSetEvent的精确度缺陷分析_备份以防后续使用到时作为参考
- mongodb 在单机上搭建分片集群 详细实战过程
- 为手机多媒体设备图像显示选择合适的协议
- PCB设计中数字地、模拟地、数字电源、模拟电源的处理方式
- Flume
- 手机流媒体技术概要
- Spring
- 自定义组件一
- 数值运算符
- 各种音频编码方式的对比