MongoDB学习之复制集

复制集原理
复制集的作用 主要为了实现mongodb服务的高可用,复制集之间是一种主从关系

  • 当数据写入时将数据迅速复制到其它节点
  • 当主节点发生故障时自动选出一个新的节点代替
由上面的特点,我们对redis了解的话,可以很快联想到哨兵模式
在除了以上的数据同步故障节点替换两大特性以外,复制集还存在以下作用
  • 数据分发:将数据分发到不同的区域,减少读的延迟
    例如公司在北京、上海、广州都有数据中心,但只有北京的是可以写入的中心,可将数据异地分发,提高上海和广州大区用户的读取效率。
  • 读写分离
  • 异地容灾:数据中心故障时快速切换到异地。
复制集的结构 复制集的结构与其它中间件的主从模式基本类似。
一个典型的复制集由3个及以上具有投票的节点组成:
  • 主节点:只有一个,负责数据的写入和读取,以及选举时投票。
  • 从节点:至少有两个,负责数据读取和选举投票。
数据的复制原理
  • 当数据(主节点)发生修改(增删改)时,它对数据的操作将被记录在一个oplog文件当中。
  • 从节点通过在主节点上代开一个tailable游标不断获取主节点中新增的oplog并且不断在自己的节点上重现这些操作,以此来实现和主节点上的数据一致。
主节点选举原理
  1. 具有投票权的节点之间两两互相发送心跳。
  2. 当5次心跳未收到时判断为节点失联。
  3. 如果主节点失联,从节点会选出新的主节点,但失联的从节点不会参与选举。
  4. 选举基于RAFT一致性算法实现,选举成功的必要条件是大多数投票节点存活。
  5. 复制集中最多可有50个节点,但有投票权的最多有7个。
  6. 被选举为主节点的的节点 必须能与多数节点建立连接、有比较新的oplog、比较高的优先级(如果配置中有)
搭建一个复制集
下面是在linux下创建一个mongodb复制集
创建三个目录
mkdir -p /data/mongodb/db{1,2,3}

配置文件
#节点1 systemLog: destination: file path: /data/mongodb/db1/mongod.log #日志路径 logAppend: true storage: dbPath: /data/mongodb/db1 #数据文件 net: bindIp: 0.0.0.0#在所有网卡监听,为了外网访问 port: 28017 replication:#复制集,没有的话就是一个单节点 replSetName: rs0 #复制集名字 processManagement: fork: true #把进程作为独立的后台进程进程 #节点2 systemLog: destination: file path: /data/mongodb/db2/mongod.log #日志路径 logAppend: true storage: dbPath: /data/mongodb/db2 #数据文件 net: bindIp: 0.0.0.0#在所有网卡监听,为了外网访问 port: 28018 replication:#复制集,没有的话就是一个单节点 replSetName: rs0 #复制集名字 processManagement: fork: true #把进程作为独立的后台进程进程 #节点3 systemLog: destination: file path: /data/mongodb/db3/mongod.log #日志路径 logAppend: true storage: dbPath: /data/mongodb/db3 #数据文件 net: bindIp: 0.0.0.0#在所有网卡监听,为了外网访问 port: 28019 replication:#复制集,没有的话就是一个单节点 replSetName: rs0 #复制集名字 processManagement: fork: true #把进程作为独立的后台进程进程

启动进程
mongod -f db1/mongod.conf mongod -f db2/mongod.conf mongod -f db3/mongod.conf

配置复制集
  1. 进入主节点
    mongo --port 28017
  2. 设置主节点
    rs.initiate()
    { "info2" : "no configuration specified. Using a default configuration for the set", "me" : "supman:28017", "ok" : 1 }

  3. 查看节点状态
    rs.status()
    { "set" : "rs0", "date" : ISODate("2021-11-30T15:33:30.616Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 1, "writeMajorityCount" : 1, "votingMembersCount" : 1, "writableVotingMembersCount" : 1, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1638286404, 1), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2021-11-30T15:33:24.656Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1638286404, 1), "t" : NumberLong(1) }, "readConcernMajorityWallTime" : ISODate("2021-11-30T15:33:24.656Z"), "appliedOpTime" : { "ts" : Timestamp(1638286404, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1638286404, 1), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2021-11-30T15:33:24.656Z"), "lastDurableWallTime" : ISODate("2021-11-30T15:33:24.656Z") }, "lastStableRecoveryTimestamp" : Timestamp(1638286364, 1), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2021-11-30T15:27:54.614Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1638286074, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 1, "priorityAtElection" : 1, "electionTimeoutMillis" : NumberLong(10000), "newTermStartDate" : ISODate("2021-11-30T15:27:54.637Z"), "wMajorityWriteAvailabilityDate" : ISODate("2021-11-30T15:27:54.660Z") }, "members" : [ { "_id" : 0, "name" : "supman:28017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 1227, "optime" : { "ts" : Timestamp(1638286404, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-30T15:33:24Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1638286074, 2), "electionDate" : ISODate("2021-11-30T15:27:54Z"), "configVersion" : 1, "configTerm" : 1, "self" : true, "lastHeartbeatMessage" : "" } ], "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1638286404, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1638286404, 1) }

    找到status信息里面的 members.name 字段
    "name" : "supman:28017"

  4. 增加从节点
    rs.add("supman:28018")
    { "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1638286626, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1638286626, 1) }

    rs.add("supman:28019")
    { "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1638286660, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1638286660, 1) }

  5. 再次查看状态
    { "set" : "rs0", "date" : ISODate("2021-11-30T15:44:29.903Z"), "myState" : 1, "term" : NumberLong(1), "syncSourceHost" : "", "syncSourceId" : -1, "heartbeatIntervalMillis" : NumberLong(2000), "majorityVoteCount" : 2, "writeMajorityCount" : 2, "votingMembersCount" : 3, "writableVotingMembersCount" : 3, "optimes" : { "lastCommittedOpTime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "lastCommittedWallTime" : ISODate("2021-11-30T15:44:24.673Z"), "readConcernMajorityOpTime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "readConcernMajorityWallTime" : ISODate("2021-11-30T15:44:24.673Z"), "appliedOpTime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "durableOpTime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "lastAppliedWallTime" : ISODate("2021-11-30T15:44:24.673Z"), "lastDurableWallTime" : ISODate("2021-11-30T15:44:24.673Z") }, "lastStableRecoveryTimestamp" : Timestamp(1638287034, 1), "electionCandidateMetrics" : { "lastElectionReason" : "electionTimeout", "lastElectionDate" : ISODate("2021-11-30T15:27:54.614Z"), "electionTerm" : NumberLong(1), "lastCommittedOpTimeAtElection" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "lastSeenOpTimeAtElection" : { "ts" : Timestamp(1638286074, 1), "t" : NumberLong(-1) }, "numVotesNeeded" : 1, "priorityAtElection" : 1, "electionTimeoutMillis" : NumberLong(10000), "newTermStartDate" : ISODate("2021-11-30T15:27:54.637Z"), "wMajorityWriteAvailabilityDate" : ISODate("2021-11-30T15:27:54.660Z") }, "members" : [ { "_id" : 0, "name" : "supman:28017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 1886, "optime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-30T15:44:24Z"), "syncSourceHost" : "", "syncSourceId" : -1, "infoMessage" : "", "electionTime" : Timestamp(1638286074, 2), "electionDate" : ISODate("2021-11-30T15:27:54Z"), "configVersion" : 3, "configTerm" : 1, "self" : true, "lastHeartbeatMessage" : "" }, { "_id" : 1, "name" : "supman:28018", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 442, "optime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-30T15:44:24Z"), "optimeDurableDate" : ISODate("2021-11-30T15:44:24Z"), "lastHeartbeat" : ISODate("2021-11-30T15:44:28.545Z"), "lastHeartbeatRecv" : ISODate("2021-11-30T15:44:28.566Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "supman:28017", "syncSourceId" : 0, "infoMessage" : "", "configVersion" : 3, "configTerm" : 1 }, { "_id" : 2, "name" : "supman:28019", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 409, "optime" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "optimeDurable" : { "ts" : Timestamp(1638287064, 1), "t" : NumberLong(1) }, "optimeDate" : ISODate("2021-11-30T15:44:24Z"), "optimeDurableDate" : ISODate("2021-11-30T15:44:24Z"), "lastHeartbeat" : ISODate("2021-11-30T15:44:28.565Z"), "lastHeartbeatRecv" : ISODate("2021-11-30T15:44:29.316Z"), "pingMs" : NumberLong(0), "lastHeartbeatMessage" : "", "syncSourceHost" : "supman:28018", "syncSourceId" : 1, "infoMessage" : "", "configVersion" : 3, "configTerm" : 1 } ], "ok" : 1, "$clusterTime" : { "clusterTime" : Timestamp(1638287064, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1638287064, 1) }

  6. 在从节点上配置可读
    进入从节点
    mongo --port 28018 mongo --port 28019
    设置从节点可读
    【MongoDB学习之复制集】rs.slaveOk()
  7. 最后在主节点增加一个用户,创建一个数据库和集合,插入数据后也能在从节点查看到数据了

    推荐阅读