佳木斯湛栽影视文化发展公司

主頁(yè) > 知識(shí)庫(kù) > 分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB分片集群的問(wèn)題

分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB分片集群的問(wèn)題

熱門標(biāo)簽:網(wǎng)站排名優(yōu)化 AI電銷 百度競(jìng)價(jià)排名 地方門戶網(wǎng)站 服務(wù)外包 Linux服務(wù)器 呼叫中心市場(chǎng)需求 鐵路電話系統(tǒng)

  前文我們聊到了mongodb的副本集以及配置副本集,回顧請(qǐng)參考  https://www.jb51.net/article/167270.htm 今天我們來(lái)聊下mongodb的分片;

  1、什么是分片?為什么要分片?

  我們知道數(shù)據(jù)庫(kù)服務(wù)器一般出現(xiàn)瓶頸是在磁盤io上,或者高并發(fā)網(wǎng)絡(luò)io,又或者單臺(tái)server的cpu、內(nèi)存等等一系列原因;于是,為了解決這些瓶頸問(wèn)題,我們就必須擴(kuò)展服務(wù)器性能;通常擴(kuò)展服務(wù)器有向上擴(kuò)展和向外擴(kuò)展;所謂向上擴(kuò)展就是給服務(wù)器加更大的磁盤,使用更大更好的內(nèi)存,更換更好的cpu;這種擴(kuò)展在一定程度上是可以解決性能瓶頸問(wèn)題,但隨著數(shù)據(jù)量大增大,瓶頸會(huì)再次出現(xiàn);所以通常這種向上擴(kuò)展的方式不推薦;向外擴(kuò)展是指一臺(tái)服務(wù)器不夠加兩臺(tái),兩臺(tái)不夠加三臺(tái),以這種方式擴(kuò)展,只要出現(xiàn)瓶頸我們就可以使用增加服務(wù)器來(lái)解決;這樣一來(lái)服務(wù)器性能解決了,但用戶的讀寫(xiě)怎么分散到多個(gè)服務(wù)器上去呢?所以我們還要想辦法把數(shù)據(jù)切分成多塊,讓每個(gè)服務(wù)器只保存整個(gè)數(shù)據(jù)集的部分?jǐn)?shù)據(jù),這樣一來(lái)使得原來(lái)一個(gè)很大的數(shù)據(jù)集就通過(guò)切片的方式,把它切分成多分,分散的存放在多個(gè)服務(wù)器上,這就是分片;分片是可以有效解決用戶寫(xiě)操作性能瓶頸;雖然解決了服務(wù)器性能問(wèn)題和用戶寫(xiě)性能問(wèn)題,同時(shí)也帶來(lái)了一個(gè)新問(wèn)題,就是用戶的查詢;我們把整個(gè)數(shù)據(jù)集分散到多個(gè)server上,那么用戶怎么查詢數(shù)據(jù)呢?比如用戶要查詢年齡大于30的用戶,該怎么查詢呢?而年齡大于30的用戶的數(shù)據(jù),可能server1上有一部分?jǐn)?shù)據(jù),server2上有部分?jǐn)?shù)據(jù),我們?cè)趺床拍軌虬阉袧M足條件的數(shù)據(jù)全部查詢到呢?這個(gè)場(chǎng)景有點(diǎn)類似我們之前說(shuō)的mogilefs的架構(gòu),用戶上傳圖片到mogilefs首先要把圖片的元數(shù)據(jù)寫(xiě)進(jìn)tracker,然后在把數(shù)據(jù)存放在對(duì)應(yīng)的data節(jié)點(diǎn),這樣一來(lái)用戶來(lái)查詢,首先找tracker節(jié)點(diǎn),tracker會(huì)把用戶的請(qǐng)求文件的元數(shù)據(jù)告訴客戶端,然后客戶端在到對(duì)應(yīng)的data節(jié)點(diǎn)取數(shù)據(jù),最后拼湊成一張圖片;而在mongodb上也是很類似,不同的的是在mogilefs上,客戶端需要自己去和后端的data節(jié)點(diǎn)交互,取出數(shù)據(jù);在mongdb上客戶端不需要直接和后端的data節(jié)點(diǎn)交互,而是通過(guò)mongodb專有的客戶端代理去代客戶端交互,最后把數(shù)據(jù)統(tǒng)一由代理返回給客戶端;這樣一來(lái)就可以解決用戶的查詢問(wèn)題;簡(jiǎn)單講所謂分片就是把一個(gè)大的數(shù)據(jù)集通過(guò)切分的方式切分成多分,分散的存放在多個(gè)服務(wù)器上;分片的目的是為了解決數(shù)據(jù)量過(guò)大而導(dǎo)致的性能問(wèn)題;

  2、數(shù)據(jù)集分片示意圖

  提示:我們通過(guò)分片,可以將原本1T的數(shù)據(jù)集,平均分成4分,每個(gè)節(jié)點(diǎn)存儲(chǔ)原有數(shù)據(jù)集的1/4,使得原來(lái)用一臺(tái)服務(wù)器處理1T的數(shù)據(jù),現(xiàn)在可以用4臺(tái)服務(wù)器來(lái)處理,這樣一來(lái)就有效的提高了數(shù)據(jù)處理過(guò)程;這也是分布式系統(tǒng)的意義;在mongodb中我們把這種共同處理一個(gè)數(shù)據(jù)集的部分?jǐn)?shù)據(jù)的節(jié)點(diǎn)叫shard,我們把使用這種分片機(jī)制的mongodb集群就叫做mongodb分片集群;

  3、mongodb分片集群架構(gòu)

  提示:在mongodb分片集群中,通常有三類角色,第一類是router角色,router角色主要用來(lái)接收客戶端的讀寫(xiě)請(qǐng)求,主要運(yùn)行mongos這個(gè)服務(wù);為了使得router角色的高可用,通常會(huì)用多個(gè)節(jié)點(diǎn)來(lái)組成router高可用集群;第二類是config server,這類角色主要用來(lái)保存mongodb分片集群中的數(shù)據(jù)和集群的元數(shù)據(jù)信息,有點(diǎn)類似mogilefs中的tracker的作用;為了保證config server的高可用性,通常config server也會(huì)將其運(yùn)行為一個(gè)副本集;第三類是shard角色,這類角色主要用來(lái)存放數(shù)據(jù),類似mogilefs的數(shù)據(jù)節(jié)點(diǎn),為了保證數(shù)據(jù)的高可用和完整性,通常每個(gè)shard是一個(gè)副本集;

  4、mongodb分片集群工作過(guò)程

  首先用戶將請(qǐng)求發(fā)送給router,router接收到用戶請(qǐng)求,然后去找config server拿對(duì)應(yīng)請(qǐng)求的元數(shù)據(jù)信息,router拿到元數(shù)據(jù)信息后,然后再向?qū)?yīng)的shard請(qǐng)求數(shù)據(jù),最后將數(shù)據(jù)整合后響應(yīng)給用戶;在這個(gè)過(guò)程中router 就相當(dāng)于mongodb的一個(gè)客戶端代理;而config server用來(lái)存放數(shù)據(jù)的元數(shù)據(jù)信息,這些信息主要包含了那些shard上存放了那些數(shù)據(jù),對(duì)應(yīng)的那些數(shù)據(jù)存放在那些shard上,和mogilefs上的tracker非常類似,主要存放了兩張表,一個(gè)是以數(shù)據(jù)為中心的一張表,一個(gè)是以shard節(jié)點(diǎn)為中心的一張表;

  5、mongodb是怎么分片的?

  在mongodb的分片集群中,分片是按照collection字段來(lái)分的,我們把指定的字段叫shard key;根據(jù)shard key的取值不同和應(yīng)用場(chǎng)景,我們可以基于shard key取值范圍來(lái)分片,也可以基于shard key做hash分片;分好片以后將結(jié)果保存在config server上;在configserver 上保存了每一個(gè)分片對(duì)應(yīng)的數(shù)據(jù)集;比如我們基于shardkey的范圍來(lái)分片,在configserver上就記錄了一個(gè)連續(xù)范圍的shardkey的值都保存在一個(gè)分片上;如下圖

  上圖主要描述了基于范圍的分片,從shardkey最小值到最大值進(jìn)行分片,把最小值到-75這個(gè)范圍值的數(shù)據(jù)塊保存在第一個(gè)分片上,把-75到25這個(gè)范圍值的數(shù)據(jù)塊保存在第二個(gè)分片上,依次類推;這種基于范圍的分片,很容易導(dǎo)致某個(gè)分片上的數(shù)據(jù)過(guò)大,而有的分片上的數(shù)據(jù)又很小,造成分片數(shù)據(jù)不均勻;所以除了基與shard key的值的范圍分片,也可以基于shard key的值做hash分片,如下圖

  基于hash分片,主要是對(duì)shardkey做hash計(jì)算后,然后根據(jù)最后的結(jié)果落在哪個(gè)分片上就把對(duì)應(yīng)的數(shù)據(jù)塊保存在對(duì)應(yīng)的分片上;比如我們把shandkey做hash計(jì)算,然后對(duì)分片數(shù)量進(jìn)行取模計(jì)算,如果得到的結(jié)果是0,那么就把對(duì)應(yīng)的數(shù)據(jù)塊保存在第一個(gè)分片上,如果取得到結(jié)果是1就保存在第二個(gè)分片上依次類推;這種基于hash分片,就有效的降低分片數(shù)據(jù)不均衡的情況,因?yàn)閔ash計(jì)算的值是散列的;

  除了上述兩種切片的方式以外,我們還可以根據(jù)區(qū)域切片,也叫基于列表切片,如下圖

  上圖主要描述了基于區(qū)域分片,這種分片一般是針對(duì)shardkey的取值范圍不是一個(gè)順序的集合,而是一個(gè)離散的集合,比如我們可用這種方式對(duì)全國(guó)省份這個(gè)字段做切片,把流量特別大的省份單獨(dú)切一個(gè)片,把流量小的幾個(gè)省份組合切分一片,把國(guó)外的訪問(wèn)或不是國(guó)內(nèi)省份的切分為一片;這種切片有點(diǎn)類似給shardkey做分類;不管用什么方式去做分片,我們盡可能的遵循寫(xiě)操作要越分散越好,讀操作要越集中越好;

  6、mongodb分片集群搭建

  環(huán)境說(shuō)明

主機(jī)名 角色 ip地址
node01 router 192.168.0.41
node02/node03/node04 config server replication set

192.168.0.42

192.168.0.43

192.168.0.44

node05/node06/node07 shard1 replication set

192.168.0.45

192.168.0.46

192.168.0.47

node08/node09/node10 shard2 replication set

192.168.0.48

192.168.0.49

192.168.0.50

  基礎(chǔ)環(huán)境,各server做時(shí)間同步,關(guān)閉防火墻,關(guān)閉selinux,ssh互信,主機(jī)名解析

  主機(jī)名解析

[root@node01 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1   localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.99 time.test.org time-node
192.168.0.41 node01.test.org node01
192.168.0.42 node02.test.org node02
192.168.0.43 node03.test.org node03
192.168.0.44 node04.test.org node04
192.168.0.45 node05.test.org node05
192.168.0.46 node06.test.org node06
192.168.0.47 node07.test.org node07
192.168.0.48 node08.test.org node08
192.168.0.49 node09.test.org node09
192.168.0.50 node10.test.org node10
192.168.0.51 node11.test.org node11
192.168.0.52 node12.test.org node12
[root@node01 ~]#

  準(zhǔn)備好基礎(chǔ)環(huán)境以后,配置mongodb yum源

[root@node01 ~]# cat /etc/yum.repos.d/mongodb.repo
[mongodb-org]
name = MongoDB Repository
baseurl = https://mirrors.aliyun.com/mongodb/yum/redhat/7/mongodb-org/4.4/x86_64/
gpgcheck = 1
enabled = 1
gpgkey = https://www.mongodb.org/static/pgp/server-4.4.asc
[root@node01 ~]#

  將mongodb yum源復(fù)制給其他節(jié)點(diǎn)

[root@node01 ~]# for i in {02..10} ; do scp /etc/yum.repos.d/mongodb.repo node$i:/etc/yum.repos.d/; done
mongodb.repo                 100% 206 247.2KB/s 00:00 
mongodb.repo                 100% 206 222.3KB/s 00:00 
mongodb.repo                 100% 206 118.7KB/s 00:00 
mongodb.repo                 100% 206 164.0KB/s 00:00 
mongodb.repo                 100% 206 145.2KB/s 00:00 
mongodb.repo                 100% 206 119.9KB/s 00:00 
mongodb.repo                 100% 206 219.2KB/s 00:00 
mongodb.repo                 100% 206 302.1KB/s 00:00 
mongodb.repo                 100% 206 289.3KB/s 00:00 
[root@node01 ~]#

  在每個(gè)節(jié)點(diǎn)上安裝mongodb-org這個(gè)包

for i in {01..10} ;
 do ssh node$i ' yum -y install mongodb-org '; 
done

  在config server 和shard節(jié)點(diǎn)上創(chuàng)建數(shù)據(jù)目錄和日志目錄,并將其屬主和屬組更改為mongod

[root@node01 ~]# for i in {02..10} ; do ssh node$i 'mkdir -p /mongodb/{data,log}  chown -R mongod.mongod /mongodb/  ls -ld /mongodb'; done
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:47 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
drwxr-xr-x 4 mongod mongod 29 Nov 11 22:45 /mongodb
[root@node01 ~]#

  配置shard1 replication set

[root@node05 ~]# cat /etc/mongod.conf
systemLog:
 destination: file
 logAppend: true
 path: /mongodb/log/mongod.log
 
storage:
 dbPath: /mongodb/data/
 journal:
 enabled: true
 
processManagement:
 fork: true
 pidFilePath: /var/run/mongodb/mongod.pid
 timeZoneInfo: /usr/share/zoneinfo
 
net:
 bindIp: 0.0.0.0
 
sharding:
 clusterRole: shardsvr
 
replication:
 replSetName: shard1_replset
[root@node05 ~]# scp /etc/mongod.conf node06:/etc/
mongod.conf                 100% 360 394.5KB/s 00:00 
[root@node05 ~]# scp /etc/mongod.conf node07:/etc/
mongod.conf                 100% 360 351.7KB/s 00:00 
[root@node05 ~]#

  配置shard2 replication set

[root@node08 ~]# cat /etc/mongod.conf
systemLog:
 destination: file
 logAppend: true
 path: /mongodb/log/mongod.log
 
storage:
 dbPath: /mongodb/data/
 journal:
 enabled: true
 
processManagement:
 fork: true
 pidFilePath: /var/run/mongodb/mongod.pid
 timeZoneInfo: /usr/share/zoneinfo
 
net:
 bindIp: 0.0.0.0
 
sharding:
 clusterRole: shardsvr
 
replication:
 replSetName: shard2_replset
[root@node08 ~]# scp /etc/mongod.conf node09:/etc/
mongod.conf                 100% 360 330.9KB/s 00:00 
[root@node08 ~]# scp /etc/mongod.conf node10:/etc/
mongod.conf                 100% 360 385.9KB/s 00:00 
[root@node08 ~]#

  啟動(dòng)shard1 replication set和shard2 replication set

[root@node05 ~]# systemctl start mongod.service
[root@node05 ~]# ss -tnl
State  Recv-Q Send-Q   Local Address:Port       Peer Address:Port    
LISTEN  0  128       *:22          *:*     
LISTEN  0  100     127.0.0.1:25          *:*     
LISTEN  0  128       *:27018         *:*     
LISTEN  0  128       :::22          :::*     
LISTEN  0  100      ::1:25          :::*     
[root@node05 ~]#for i in {06..10} ; do ssh node$i 'systemctl start mongod.service  ss -tnl';done
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   *:27018     *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   *:27018     *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   *:27018     *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   *:27018     *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   *:27018     *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
[root@node05 ~]#

  提示:默認(rèn)不指定shard監(jiān)聽(tīng)端口,它默認(rèn)就監(jiān)聽(tīng)在27018端口,所以啟動(dòng)shard節(jié)點(diǎn)后,請(qǐng)確保27018端口正常監(jiān)聽(tīng)即可;

  連接node05的mongodb 初始化shard1_replset副本集

> rs.initiate(
... {
...  _id : "shard1_replset",
...  members: [
...  { _id : 0, host : "node05:27018" },
...  { _id : 1, host : "node06:27018" },
...  { _id : 2, host : "node07:27018" }
...  ]
... }
... )
{
  "ok" : 1,
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605107401, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  },
  "operationTime" : Timestamp(1605107401, 1)
}
shard1_replset:SECONDARY>

  連接node08的mongodb 初始化shard2_replset副本集

> rs.initiate(
... {
...  _id : "shard2_replset",
...  members: [
...  { _id : 0, host : "node08:27018" },
...  { _id : 1, host : "node09:27018" },
...  { _id : 2, host : "node10:27018" }
...  ]
... }
... )
{
  "ok" : 1,
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605107644, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  },
  "operationTime" : Timestamp(1605107644, 1)
}
shard2_replset:OTHER>

  配置configserver replication set

[root@node02 ~]# cat /etc/mongod.conf
systemLog:
 destination: file
 logAppend: true
 path: /mongodb/log/mongod.log
 
storage:
 dbPath: /mongodb/data/
 journal:
 enabled: true
 
processManagement:
 fork: true
 pidFilePath: /var/run/mongodb/mongod.pid
 timeZoneInfo: /usr/share/zoneinfo
 
net:
 bindIp: 0.0.0.0
 
sharding:
 clusterRole: configsvr
 
replication:
 replSetName: cfg_replset
[root@node02 ~]# scp /etc/mongod.conf node03:/etc/mongod.conf
mongod.conf                 100% 358 398.9KB/s 00:00 
[root@node02 ~]# scp /etc/mongod.conf node04:/etc/mongod.conf 
mongod.conf                 100% 358 270.7KB/s 00:00 
[root@node02 ~]#

  啟動(dòng)config server

[root@node02 ~]# systemctl start mongod.service
[root@node02 ~]# ss -tnl
State  Recv-Q Send-Q   Local Address:Port       Peer Address:Port    
LISTEN  0  128       *:27019         *:*     
LISTEN  0  128       *:22          *:*     
LISTEN  0  100     127.0.0.1:25          *:*     
LISTEN  0  128       :::22          :::*     
LISTEN  0  100      ::1:25          :::*     
[root@node02 ~]# ssh node03 'systemctl start mongod.service  ss -tnl' 
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:27019     *:*     
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
[root@node02 ~]# ssh node04 'systemctl start mongod.service  ss -tnl'
State  Recv-Q Send-Q Local Address:Port    Peer Address:Port    
LISTEN  0  128   *:27019     *:*     
LISTEN  0  128   *:22      *:*     
LISTEN  0  100 127.0.0.1:25      *:*     
LISTEN  0  128   :::22      :::*     
LISTEN  0  100  ::1:25      :::*     
[root@node02 ~]#

  提示:config server 默認(rèn)在不指定端口的情況監(jiān)聽(tīng)在27019這個(gè)端口,啟動(dòng)后,請(qǐng)確保該端口處于正常監(jiān)聽(tīng);

  連接node02的mongodb,初始化cfg_replset 副本集

> rs.initiate(
... {
...  _id: "cfg_replset",
...  configsvr: true,
...  members: [
...  { _id : 0, host : "node02:27019" },
...  { _id : 1, host : "node03:27019" },
...  { _id : 2, host : "node04:27019" }
...  ]
... }
... )
{
  "ok" : 1,
  "$gleStats" : {
    "lastOpTime" : Timestamp(1605108177, 1),
    "electionId" : ObjectId("000000000000000000000000")
  },
  "lastCommittedOpTime" : Timestamp(0, 0),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605108177, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  },
  "operationTime" : Timestamp(1605108177, 1)
}
cfg_replset:SECONDARY>

  配置router

[root@node01 ~]# cat /etc/mongos.conf
systemLog:
 destination: file
 path: /var/log/mongodb/mongos.log
 logAppend: true
 
processManagement:
 fork: true
 
net:
 bindIp: 0.0.0.0
sharding:
 configDB: "cfg_replset/node02:27019,node03:27019,node04:27019"
[root@node01 ~]#

  提示:configDB必須是副本集名稱/成員監(jiān)聽(tīng)地址:port的形式,成員至少要寫(xiě)一個(gè);

  啟動(dòng)router

[root@node01 ~]# mongos -f /etc/mongos.conf
about to fork child process, waiting until server is ready for connections.
forked process: 1510
child process started successfully, parent exiting
[root@node01 ~]# ss -tnl
State  Recv-Q Send-Q   Local Address:Port       Peer Address:Port    
LISTEN  0  128       *:22          *:*     
LISTEN  0  100     127.0.0.1:25          *:*     
LISTEN  0  128       *:27017         *:*     
LISTEN  0  128       :::22          :::*     
LISTEN  0  100      ::1:25          :::*     
[root@node01 ~]#

  連接mongos,添加shard1 replication set 和shard2 replication set

mongos> sh.addShard("shard1_replset/node05:27018,node06:27018,node07:27018")
{
  "shardAdded" : "shard1_replset",
  "ok" : 1,
  "operationTime" : Timestamp(1605109085, 3),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605109086, 1),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  }
}
mongos> sh.addShard("shard2_replset/node08:27018,node09:27018,node10:27018")
{
  "shardAdded" : "shard2_replset",
  "ok" : 1,
  "operationTime" : Timestamp(1605109118, 2),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605109118, 3),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  }
}
mongos>

  提示:添加shard 副本集也是需要指明副本集名稱/成員的格式添加;

  到此分片集群就配置好了

  查看sharding 集群狀態(tài)

mongos> sh.status()
--- Sharding Status ---
 sharding version: {
  "_id" : 1,
  "minCompatibleVersion" : 5,
  "currentVersion" : 6,
  "clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
 }
 shards:
  { "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
  { "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
 active mongoses:
  "4.4.1" : 1
 autosplit:
  Currently enabled: yes
 balancer:
  Currently enabled: yes
  Currently running: yes
  Collections with active migrations:
    config.system.sessions started at Wed Nov 11 2020 23:43:14 GMT+0800 (CST)
  Failed balancer rounds in last 5 attempts: 0
  Migration Results for the last 24 hours:
    45 : Success
 databases:
  { "_id" : "config", "primary" : "config", "partitioned" : true }
    config.system.sessions
      shard key: { "_id" : 1 }
      unique: false
      balancing: true
      chunks:
        shard1_replset 978
        shard2_replset 46
      too many chunks to print, use verbose if you want to force print
mongos>

  提示:可以看到當(dāng)前分片集群中有兩個(gè)shard 副本集,分別是shard1_replset和shard2_replset;以及一個(gè)config server

  對(duì)testdb數(shù)據(jù)庫(kù)啟用sharding功能

mongos> sh.enableSharding("testdb")
{
  "ok" : 1,
  "operationTime" : Timestamp(1605109993, 9),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605109993, 9),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  }
}
mongos> sh.status()
--- Sharding Status ---
 sharding version: {
  "_id" : 1,
  "minCompatibleVersion" : 5,
  "currentVersion" : 6,
  "clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
 }
 shards:
  { "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
  { "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
 active mongoses:
  "4.4.1" : 1
 autosplit:
  Currently enabled: yes
 balancer:
  Currently enabled: yes
  Currently running: no
  Failed balancer rounds in last 5 attempts: 0
  Migration Results for the last 24 hours:
    214 : Success
 databases:
  { "_id" : "config", "primary" : "config", "partitioned" : true }
    config.system.sessions
      shard key: { "_id" : 1 }
      unique: false
      balancing: true
      chunks:
        shard1_replset 810
        shard2_replset 214
      too many chunks to print, use verbose if you want to force print
  { "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
mongos>

  提示:在對(duì)某個(gè)數(shù)據(jù)庫(kù)啟動(dòng)sharding功能后,它會(huì)給我們分片一個(gè)主shard所謂主shard是用來(lái)存放該數(shù)據(jù)庫(kù)下沒(méi)有做分片的colleciton;對(duì)于分片的collection會(huì)分散在各個(gè)shard上;

  啟用對(duì)testdb庫(kù)下的peoples集合啟動(dòng)sharding,并指明在age字段上做基于范圍的分片

mongos> sh.shardCollection("testdb.peoples",{"age":1})
{
  "collectionsharded" : "testdb.peoples",
  "collectionUUID" : UUID("ec095411-240d-4484-b45d-b541c33c3975"),
  "ok" : 1,
  "operationTime" : Timestamp(1605110694, 11),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605110694, 11),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  }
}
mongos> sh.status()
--- Sharding Status ---
 sharding version: {
  "_id" : 1,
  "minCompatibleVersion" : 5,
  "currentVersion" : 6,
  "clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
 }
 shards:
  { "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
  { "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
 active mongoses:
  "4.4.1" : 1
 autosplit:
  Currently enabled: yes
 balancer:
  Currently enabled: yes
  Currently running: no
  Failed balancer rounds in last 5 attempts: 0
  Migration Results for the last 24 hours:
    408 : Success
 databases:
  { "_id" : "config", "primary" : "config", "partitioned" : true }
    config.system.sessions
      shard key: { "_id" : 1 }
      unique: false
      balancing: true
      chunks:
        shard1_replset 616
        shard2_replset 408
      too many chunks to print, use verbose if you want to force print
  { "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
    testdb.peoples
      shard key: { "age" : 1 }
      unique: false
      balancing: true
      chunks:
        shard2_replset 1
      { "age" : { "$minKey" : 1 } } -->> { "age" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 0)
mongos>

  提示:如果對(duì)應(yīng)的collection存在,我們還需要先對(duì)collection創(chuàng)建shardkey索引,然后在使用sh.shardCollection()來(lái)對(duì)colleciton啟用sharding功能;基于范圍做分片,我們可以在多個(gè)字段上做;

  基于hash做分片

mongos> sh.shardCollection("testdb.peoples1",{"name":"hashed"})
{
  "collectionsharded" : "testdb.peoples1",
  "collectionUUID" : UUID("f6213da1-7c7d-4d5e-8fb1-fc554efb9df2"),
  "ok" : 1,
  "operationTime" : Timestamp(1605111014, 2),
  "$clusterTime" : {
    "clusterTime" : Timestamp(1605111014, 2),
    "signature" : {
      "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
      "keyId" : NumberLong(0)
    }
  }
}
mongos> sh.status()
--- Sharding Status ---
 sharding version: {
  "_id" : 1,
  "minCompatibleVersion" : 5,
  "currentVersion" : 6,
  "clusterId" : ObjectId("5fac01dd8d6fa3fe899662c8")
 }
 shards:
  { "_id" : "shard1_replset", "host" : "shard1_replset/node05:27018,node06:27018,node07:27018", "state" : 1 }
  { "_id" : "shard2_replset", "host" : "shard2_replset/node08:27018,node09:27018,node10:27018", "state" : 1 }
 active mongoses:
  "4.4.1" : 1
 autosplit:
  Currently enabled: yes
 balancer:
  Currently enabled: yes
  Currently running: yes
  Collections with active migrations:
    config.system.sessions started at Thu Nov 12 2020 00:10:16 GMT+0800 (CST)
  Failed balancer rounds in last 5 attempts: 0
  Migration Results for the last 24 hours:
    480 : Success
 databases:
  { "_id" : "config", "primary" : "config", "partitioned" : true }
    config.system.sessions
      shard key: { "_id" : 1 }
      unique: false
      balancing: true
      chunks:
        shard1_replset 543
        shard2_replset 481
      too many chunks to print, use verbose if you want to force print
  { "_id" : "testdb", "primary" : "shard2_replset", "partitioned" : true, "version" : { "uuid" : UUID("454aad2e-b397-4c88-b5c4-c3b21d37e480"), "lastMod" : 1 } }
    testdb.peoples
      shard key: { "age" : 1 }
      unique: false
      balancing: true
      chunks:
        shard2_replset 1
      { "age" : { "$minKey" : 1 } } -->> { "age" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 0)
    testdb.peoples1
      shard key: { "name" : "hashed" }
      unique: false
      balancing: true
      chunks:
        shard1_replset 2
        shard2_replset 2
      { "name" : { "$minKey" : 1 } } -->> { "name" : NumberLong("-4611686018427387902") } on : shard1_replset Timestamp(1, 0)
      { "name" : NumberLong("-4611686018427387902") } -->> { "name" : NumberLong(0) } on : shard1_replset Timestamp(1, 1)
      { "name" : NumberLong(0) } -->> { "name" : NumberLong("4611686018427387902") } on : shard2_replset Timestamp(1, 2)
      { "name" : NumberLong("4611686018427387902") } -->> { "name" : { "$maxKey" : 1 } } on : shard2_replset Timestamp(1, 3)
mongos>

  提示:基于hash做分片只能在一個(gè)字段上做,不能指定多個(gè)字段;從上面的狀態(tài)信息可以看到testdb.peoples被分到了shard2上,peoples1一部分分到了shard1,一部分分到了shard2上;所以在peoples中插入多少條數(shù)據(jù),它都會(huì)寫(xiě)到shard2上,在peoples1中插入數(shù)據(jù)會(huì)被寫(xiě)入到shard1和shard2上;

  驗(yàn)證:在peoples1 集合上插入數(shù)據(jù),看看是否將數(shù)據(jù)分片到不同的shard上呢?

  在mongos上插入數(shù)據(jù)

mongos> use testdb
switched to db testdb
mongos> for (i=1;i=10000;i++) db.peoples1.insert({name:"people"+i,age:(i%120),classes:(i%20)})
WriteResult({ "nInserted" : 1 })
mongos>

  在shard1上查看數(shù)據(jù)

shard1_replset:PRIMARY> show dbs
admin 0.000GB
config 0.001GB
local 0.001GB
testdb 0.000GB
shard1_replset:PRIMARY> use testdb
switched to db testdb
shard1_replset:PRIMARY> show tables
peoples1
shard1_replset:PRIMARY> db.peoples1.find().count()
4966
shard1_replset:PRIMARY>

 提示:在shard1上可以看到對(duì)應(yīng)collection保存了4966條數(shù)據(jù);

  在shard2上查看數(shù)據(jù)

shard2_replset:PRIMARY> show dbs
admin 0.000GB
config 0.001GB
local 0.011GB
testdb 0.011GB
shard2_replset:PRIMARY> use testdb
switched to db testdb
shard2_replset:PRIMARY> show tables
peoples
peoples1
shard2_replset:PRIMARY> db.peoples1.find().count()
5034
shard2_replset:PRIMARY>

  提示:在shard2上可以看到有peoples集合和peoples1集合,其中peoples1集合保存了5034條數(shù)據(jù);shard1和shard2總共就保存了我們剛才插入的10000條數(shù)據(jù);

  ok,到此mongodb的分片集群就搭建,測(cè)試完畢了;

到此這篇關(guān)于分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB分片集群的文章就介紹到這了,更多相關(guān)MongoDB分片集群內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家!

您可能感興趣的文章:
  • 分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB訪問(wèn)控制的操作方法
  • 分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB備份與恢復(fù)的實(shí)踐詳解
  • SpringDataMongoDB多文檔事務(wù)的實(shí)現(xiàn)
  • mongodb如何對(duì)文檔內(nèi)數(shù)組進(jìn)行過(guò)濾的方法步驟
  • MongoDB中文檔的更新操作示例詳解
  • MongoDB數(shù)據(jù)庫(kù)文檔操作方法(必看篇)
  • mongodb 數(shù)據(jù)類型(null/字符串/數(shù)字/日期/內(nèi)嵌文檔/數(shù)組等)
  • PHP庫(kù) 查詢Mongodb中的文檔ID的方法
  • MongoDB如何更新多級(jí)文檔的數(shù)據(jù)

標(biāo)簽:湘潭 銅川 仙桃 蘭州 衡水 崇左 黃山 湖南

巨人網(wǎng)絡(luò)通訊聲明:本文標(biāo)題《分布式文檔存儲(chǔ)數(shù)據(jù)庫(kù)之MongoDB分片集群的問(wèn)題》,本文關(guān)鍵詞  ;如發(fā)現(xiàn)本文內(nèi)容存在版權(quán)問(wèn)題,煩請(qǐng)?zhí)峁┫嚓P(guān)信息告之我們,我們將及時(shí)溝通與處理。本站內(nèi)容系統(tǒng)采集于網(wǎng)絡(luò),涉及言論、版權(quán)與本站無(wú)關(guān)。
  • 相關(guān)文章
  • 收縮
    • 微信客服
    • 微信二維碼
    • 電話咨詢

    • 400-1100-266
    格尔木市| 金华市| 辽宁省| 山阴县| 兴仁县| 丘北县| 潮州市| 巫山县| 温州市| 宁河县| 青海省| 瑞金市| 河间市| 乐陵市| 庆城县| 鹤山市| 延寿县| 新乡市| 察雅县| 武威市| 敦化市| 丰都县| 凤城市| 辽中县| 镶黄旗| 利川市| 谷城县| 黑水县| 陇川县| 都江堰市| 商丘市| 昌图县| 太原市| 彭阳县| 信宜市| 桐城市| 甘德县| 彭山县| 开原市| 大冶市| 区。|