mongoDB-3.x Sharding with Replica

mongoDB-3.x Sharding with Replica

官方文档:

horizontal scaling(CPU,RAM,I/O)
角色:

Sharded cluster has the following components: shards, query routers and config servers.

Shards store the data. To provide high availability and data consistency, in a production sharded cluster, each shard is a replica set [1]. For more information on replica sets, see Replica Sets.

Query Routers, or mongos instances, interface with client applications and direct operations to the appropriate shard or shards. A client sends requests to a mongos, which then routes the operations to the shards and returns the results to the clients. A sharded cluster can contain more than one mongos to divide the client request load, and most sharded clusters have more than one mongos for this reason.

Config servers store the cluster’s metadata. This data contains a mapping of the cluster’s data set to the shards. The query router uses this metadata to target operations to specific shards.

Changed in version 3.2: Starting in MongoDB 3.2, config servers for sharded clusters can be deployed as areplica set. The replica set config servers must run the WiredTiger storage engine. MongoDB 3.2 deprecates the use of three mirrored mongod instances for config servers.

模型：
Diagram of a sample sharded cluster for production purposes. Contains exactly 3 config servers, 2 or more ``mongos`` query routers, and at least 2 shards. The shards are replica sets.


部署：
Replica Set请参看mongoDB-3.x Replica Set集群
Config Server Replica Set
Config-p: 192.168.192.10:27019
Config-s1: 192.168.192.20:27019
Config-s2: 192.168.192.30:27019
New in version 3.2: Starting in 3.2, config servers for a sharded cluster can be deployed as a replica set.
注意: 从3.2版本开始,Config Server可以配置为Replica, 需要至少3节点
两节点实测, 任意config节点宕机,整个Replica就变为只读(SECONDARY),所以需要至少3节点 
mongod --configsvr --replSet configReplSet --port 27019 --fork --httpinterface --rest --jsonp --setParameter enableLocalhostAuthBypass=0 --pidfilepath /opt/mongodb/mongod.pid --dbpath /opt/mongodb/db --logpath /opt/mongodb/log/mongod.log  --logappend --logRotate rename --timeStampFormat ctime
或者,配置文件中加入
replication:
  oplogSizeMB: 5120
  replSetName: configReplSet
  secondaryIndexPrefetch: all
  enableMajorityReadConcern: false
 
sharding:
  clusterRole: configsvr

配置config Repica Set
进入任意一台config server,登录mongo shell
rs.initiate( {
   _id: "configReplSet",
   configsvr: true,
   members: [
      { _id: 0, host: "192.168.192.10:27019" },
      { _id: 1, host: "192.168.192.20:27019" },
     { _id: 1, host: "192.168.192.30:27019" }
   ]
} )

Shard Server Replica Set
Shard-p1: 192.168.192.11:27018
Shard-s1: 192.168.192.21:27018
Shard-p2: 192.168.192.12:27018
Shard-s2: 192.168.192.22:27018
这里做了两组shard(Replica Set)
1.配置文件(所有节点)
replication:
  oplogSizeMB: 5120
  replSetName: rs0
  secondaryIndexPrefetch: all 
  enableMajorityReadConcern: false
sharding:
  clusterRole: shardsvr
或 命令行参数
--oplogSize 5120 --replSet rs0 --shardsvr
2.配置shard1(Replica Set)
进入mongo shell
rsconfig = {_id: 'rs0', members: [
{_id: 0, host: '192.168.192.11:27018',priority:1},
{_id: 1, host: '192.168.192.21:27018'}]
}
rs.initiate(rsconfig)
rs.conf()
3.配置shard2(Replica Set)
注:Replica名称需要更换,如 replSetName: rs1
进入mongo shell
rsconfig = {_id: 'rs1', members: [
{_id: 0, host: '192.168.192.12:27018',priority:1},
{_id: 1, host: '192.168.192.22:27018'}]
}
rs.initiate(rsconfig)
rs.conf()

mongos Instances
Router-1: 192.168.192.10:27017
Router-2: 192.168.192.20:27017
说明:mongos只作分发,并实时或后台不断balance,使整个shard趋于负载均衡,不存数据,非常轻量,可以作为单独节点,也可以和其它mongo节点放在一起,这里和config主机放在一起,生产环境最好单独放在不同的主机上
1.配置启动mongos
mongos --configdb configReplSet/192.168.192.10:27019,192.168.192.20:27019 --port 27017 --localThreshold 15 --fork --httpinterface --jsonp --setParameter enableLocalhostAuthBypass=0 --pidfilepath /opt/mongodb/mongos.pid --logpath /opt/mongodb/log/mongos.log  --logappend --logRotate rename --timeStampFormat ctime
或者,配置文件中加入
systemLog:
  destination: file
  logAppend: true
  logRotate: rename
  timeStampFormat: ctime
  path: /opt/mongodb/log/mongos.log
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /opt/mongodb/mongos.pid  # location of pidfile
net:
  port: 27017
  #bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on all interfaces.
  http:
    enabled: true
setParameter:
  enableLocalhostAuthBypass: 0
  authenticationMechanisms: MONGODB-CR
replication:
  localPingThresholdMs: 15
sharding:
  autoSplit: true
  configDB: configReplSet/192.168.192.10:27019,192.168.192.20:27019
init脚本
chown -R mongod: /opt/mongodb-linux-x86_64-rhel62-3.2.0
cp /usr/local/src/mongodb-src-r3.2.0/rpm/init.d-mongod /etc/init.d/mongos
chmod +x /etc/init.d/mongos
sed -i '/CONFIGFILE=/i MONGOD="/opt/mongodb/bin/mongos"' /etc/init.d/mongos
sed -i '/CONFIGFILE=/s:/etc/mongod.conf:/opt/mongodb/conf/mongos.conf:g' /etc/init.d/mongos
sed -i '/Starting mongod:/ s/mongod/mongos/' /etc/init.d/mongos 
sed -i '/Stopping mongod:/ s/mongod/mongos/' /etc/init.d/mongos 
service mongos start
chkconig mongos on
2.添加shards到集群
sh.addShard('rs0/192.168.192.11:27018,192.168.192.21:27018')
sh.addShard('rs1/192.168.192.12:27018,192.168.192.22:27018')
或
db.runCommand({addshard: 'rs0/192.168.192.11:27018,192.168.192.21:27018'})
db.runCommand({addshard: 'rs1/192.168.192.12:27018,192.168.192.22:27018'})
3.启用数据库级的sharding
use admin
db.runCommand({enableSharding: 'os'})
说明:启用分片后,配置信息会存放在config server的数据库config的databases集合里
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica

4.测试插入数据
这里插入了600000条数据,要本地虚拟机大概5分钟插入完成
a.递增片键方式
use admin
db.runCommand({shardcollection:"os.users",key:{_id:1}})
use os
for(var i=1; i<=600000; i++) db.users.insert({nu:i,type: 'windows',release: 'win10',company: 'MicroSoft'})
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
可以看到,分片成功,但并不一定十分均匀
或
use test
sh.enableSharding( "test" )
db.test_collection.createIndex( { number : 1 } )
sh.shardCollection( "test.test_collection", { "number" : 1 } )
var bulk = db.test_collection.initializeUnorderedBulkOp();
people = ["Marc", "Bill", "George", "Eliot", "Matt", "Trey", "Tracy", "Greg", "Steve", "Kristina", "Katie", "Jeff"];
for(var i=0; i<1000000; i++){
   user_id = i;
   name = people[Math.floor(Math.random()*people.length)];
   number = Math.floor(Math.random()*10001);
   bulk.insert( { "user_id":user_id, "name":name, "number":number });
}
bulk.execute();
db.stats()
db.printShardingStatus()
b.随机片键方式
use admin
db.runCommand({enableSharding: 'mydb'})
db.runCommand({shardcollection:"mydb.peoples",key:{ram:1}})
use mydb
for(var i=1; i<=600000; i++) db.peoples.insert({name:'foo',age:i,addr:'shanghai',country:"China", ram : Math.random()})
5.测试回收数据,移除shard1(rs0)
use admin
db.runCommand({removeshard: 'rs0/192.168.192.11:27018,192.168.192.21:27018'})
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
 可以看到, shard移除命令执行后, 集群会将rs0里的数据会陆续移到rs1中,整个过程对客户端是透明的,不影响任何读写操作,因为要移除的是PRIMARY shard所以,要等数据全部移动完成后,再次执行removeshard动作
db.runCommand({removeshard: 'rs0/192.168.192.11:27018,192.168.192.21:27018'})

注意:如果是移除Primary shard,则先需要手动移动数据,解除依赖关系后,再执行删除动作,如:
use admin
db.runCommand({moveprimary: 'os',to: 'rs1/192.168.192.12:27018,192.168.192.22:27018'})
db.runCommand({moveprimary: 'mydb',to: 'rs1/192.168.192.12:27018,192.168.192.22:27018'})
db.runCommand({removeshard: 'rs0/192.168.192.11:27018,192.168.192.21:27018'})
5.查看shard状态
use os
db.stats()
db.printShardingStatus()
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica

6.再次加入rs0
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
当移除完成后,再次加入可以能会报如某些数据库己经存在的错,使得加入shard不成功
可以先删除冗余移留的数据库
show dbs
use os
db.dropDatabase()
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica
再次加入就成功了
mongoDB-3.x <wbr>Sharding <wbr>with <wbr>Replica


【推广】 免费学中医，健康全家人
原文地址：https://www.cnblogs.com/lixuebin/p/10814245.html