【MQ-Apache|RocketMQ-初体验RocketMQ(03)_RocketMQ多机集群部署【MQ-Apache|RocketMQ】

文章目录

环境准备
参考配置
集群搭建

130主机

broker 配置文件
启动namesrv
启动broker Master 和 broker Slave

131主机

broker 配置文件
启动namesrv
启动broker Master 和 broker Slave

查看集群监控状态
测试发送和消费消息

双机互为主备的配置
RocketMQ4.3.X配置参数

【MQ-Apache|RocketMQ-初体验RocketMQ(03)_RocketMQ多机集群部署

文章图片

完成了单节点的RocketMQ的安装 RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装，我们来搞个2个节点的集群来玩下
环境准备 CentOS7
192.168.18.130 、192.168.18.131
Q: 生产环境，假设你有2台主机，应该如何部署RocketMQ集群更HA一些？
A：

文章图片

你有 A、B两台主机，
A主机部署： broker-a 主节点和 broker-b 从节点
B主机部署： broker-b 主节点和 borker-a 从节点
两台主机互为主备，HA更可靠一些，即使有一台主机，假设B主机宕机了。我们的A主机上仍然还有是 broker-a 主节点和 broker-b 从节点，这个时候只是 broker-b 只能提供消费消息的能力了，不能写入了。还是要比整个 broker-b cluster 因为主机B的宕机而全部不能提供服务了好。
参考配置在RocketMQ 安装目录 conf目录下，官方提供了一些参考配置

文章图片

可以集合自己的场景，到对应的目录下看下官方推荐的配置。
集群搭建 130 ： brokera-m brokera-s
131: brokerb-m brokerb-s
broker集群在同一台主机上互为主备。
每台机器上启动一个namesrv。两个 broker节点，互为主备，构成一个broker集群。
130主机 broker 配置文件
cp 两个文件出来

文章图片

broker-m.conf master节点的配置文件
broker-s.conf slave节点的配置文件
【【MQ-Apache|RocketMQ-初体验RocketMQ(03)_RocketMQ多机集群部署】broker-m.conf

# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements.See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License.You may obtain a copy of the License at # #http://www.apache.org/licenses/LICENSE-2.0 # #Unless required by applicable law or agreed to in writing, software #distributed under the License is distributed on an "AS IS" BASIS, #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #See the License for the specific language governing permissions and #limitations under the License.#集群名称，可自定义 brokerClusterName=DefaultCluster brokerName=broker‐a # 0 表示主节点 brokerId=0 # rocketmq‐name服务地址,多个地址用; 分开，不配置默认为localhost:9876 namesrvAddr=192.168.18.130:9876; 192.168.18.131:9876 deleteWhen=04 fileReservedTime=48 #当前节点角色 brokerRole=SYNC_MASTER flushDiskType=ASYNC_FLUSH autoCreateTopicEnable=true #broker通信端口，默认端口 listenPort=10911 #消息存储根路径 storePathRootDir=/data/rocketmq/store-m

broker-s.conf

# Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements.See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License.You may obtain a copy of the License at # #http://www.apache.org/licenses/LICENSE-2.0 # #Unless required by applicable law or agreed to in writing, software #distributed under the License is distributed on an "AS IS" BASIS, #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #See the License for the specific language governing permissions and #limitations under the License.#集群名称，可自定义 brokerClusterName=DefaultCluster brokerName=broker‐a # 非0 表示从节点 brokerId=1 # rocketmq‐name服务地址,多个地址用; 分开，不配置默认为localhost:9876 namesrvAddr=192.168.18.130:9876; 192.168.18.131:9876 deleteWhen=04 fileReservedTime=48 #当前节点角色 brokerRole=SLAVE flushDiskType=ASYNC_FLUSH autoCreateTopicEnable=true #broker通信端口，默认端口 listenPort=10811 #消息存储根路径 storePathRootDir=/data/rocketmq/store-s

主要注意的地方

brokerName 主从节点保持一致
brokerId 0 表示主节点，从节点 1,2,3…依次类推
namesrvAddr 配置所有 namesrv的地址，多个地址使用 ; 分开
brokerRole 这个要注意主节点 SYNC_MASTER 或者 ASYNC_MASTER 从节点 SLAVE
listenPort 确定端口不要重复占用，否则启动失败

启动namesrv
nohup sh bin/mqnamesrv -n 192.168.18.130:9876& 最好通过-n指定主IP，否则的话有可能在多网卡或者docker环境下启动失败

[root@artisan rocketmq-all-4.3.2-bin-release]# pwd /usr/local/rocketmq/rocketmq-all-4.3.2-bin-release [root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqnamesrv -n 192.168.18.130:9876& [1] 9236 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’ tail -f nohup.out日志如下： Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release. The Name Server boot success. serializeType=JSON

查看进程

[root@artisan rocketmq-all-4.3.2-bin-release]# jps |grep -v Jps 9239 NamesrvStartup [root@artisan rocketmq-all-4.3.2-bin-release]#

启动broker Master 和 broker Slave
主节点： nohup sh bin/mqbroker -c conf/broker-m.conf & 从节点： nohup sh bin/mqbroker -c conf/broker-s.conf &

[root@artisan rocketmq-all-4.3.2-bin-release]# >nohup.out主节点启动 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -cconf/broker-m.conf & [2] 9415 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’ tailf nohup.out The broker[brokeraa, 192.168.18.130:10911] boot success. serializeType=JSON and name server is 192.168.18.130:9876; 192.168.18.131:9876^C^C从节点启动 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -cconf/broker-s.conf & [3] 9495 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’ tailf nohup.out The broker[brokeraa, 192.168.18.130:10911] boot success. serializeType=JSON and name server is 192.168.18.130:9876; 192.168.18.131:9876The broker[brokeraa, 192.168.18.130:10811] boot success. serializeType=JSON and name server is 192.168.18.130:9876; 192.168.18.131:9876^C 查看进程 [root@artisan rocketmq-all-4.3.2-bin-release]# jps |grep -v Jps 9239 NamesrvStartup 9419 BrokerStartup 9499 BrokerStartup [root@artisan rocketmq-all-4.3.2-bin-release]#

131主机 broker 配置文件
操作同130 ，唯一不同的就是这个 brokerName 。
这里贴下配置
主节点

[root@artisan conf]# cat broker-m.conf # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements.See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License.You may obtain a copy of the License at # #http://www.apache.org/licenses/LICENSE-2.0 # #Unless required by applicable law or agreed to in writing, software #distributed under the License is distributed on an "AS IS" BASIS, #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #See the License for the specific language governing permissions and #limitations under the License.#集群名称，可自定义 brokerClusterName=DefaultCluster brokerName=broker‐b # 0 表示主节点 brokerId=0 # rocketmq‐name服务地址,多个地址用; 分开，不配置默认为localhost:9876 namesrvAddr=192.168.18.130:9876; 192.168.18.131:9876 deleteWhen=04 fileReservedTime=48 #当前节点角色 brokerRole=SYNC_MASTER flushDiskType=ASYNC_FLUSH autoCreateTopicEnable=true #broker通信端口，默认端口 listenPort=10911 #消息存储根路径 storePathRootDir=/data/rocketmq/store-m

从节点

[root@artisan conf]# cat broker-s.conf # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements.See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License.You may obtain a copy of the License at # #http://www.apache.org/licenses/LICENSE-2.0 # #Unless required by applicable law or agreed to in writing, software #distributed under the License is distributed on an "AS IS" BASIS, #WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. #See the License for the specific language governing permissions and #limitations under the License.#集群名称，可自定义 brokerClusterName=DefaultCluster brokerName=broker‐b # 非0 表示从节点 brokerId=1 # rocketmq‐name服务地址,多个地址用; 分开，不配置默认为localhost:9876 namesrvAddr=192.168.18.130:9876; 192.168.18.131:9876 deleteWhen=04 fileReservedTime=48 #当前节点角色 brokerRole=SLAVE flushDiskType=ASYNC_FLUSH autoCreateTopicEnable=true #broker通信端口，默认端口 listenPort=10811 #消息存储根路径 storePathRootDir=/data/rocketmq/store-s [root@artisan conf]#

启动namesrv
同130
启动broker Master 和 broker Slave
同130
查看集群监控状态 mqadmin clusterlist -n 192.168.18.130:9876

[root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/mqadmin clusterlist -n 192.168.18.130:9876 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0 #Cluster Name#Broker Name#BID#Addr#Version#InTPS(LOAD)#OutTPS(LOAD) #PCWait(ms) #Hour #SPACE DefaultClusterbrokeraa0192.168.18.130:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1331 DefaultClusterbrokeraa1192.168.18.130:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1331 DefaultClusterbrokerab0192.168.18.131:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1654 DefaultClusterbrokerab1192.168.18.131:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1654 ^C^C^C[root@artisan rocketmq-all-4.3.2-bin-release]# ^C [root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/mqadmin clusterlist -n 192.168.18.131:9876 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0 #Cluster Name#Broker Name#BID#Addr#Version#InTPS(LOAD)#OutTPS(LOAD) #PCWait(ms) #Hour #SPACE DefaultClusterbrokeraa0192.168.18.130:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1331 DefaultClusterbrokeraa1192.168.18.130:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1331 DefaultClusterbrokerab0192.168.18.131:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1654 DefaultClusterbrokerab1192.168.18.131:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436943.73 0.1654 [root@artisan rocketmq-all-4.3.2-bin-release]#

如果碰到了如下错误

[root@artisan bin]# ./mqadmin clusterlist -n 192.168.18.130:9876 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0 org.apache.rocketmq.tools.command.SubCommandException: ClusterListSubCommand command failed at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:93) at org.apache.rocketmq.tools.command.MQAdminStartup.main0(MQAdminStartup.java:132) at org.apache.rocketmq.tools.command.MQAdminStartup.main(MQAdminStartup.java:83) Caused by: org.apache.rocketmq.remoting.exception.RemotingTimeoutException: wait response on the channel <192.168.18.130:9876> timeout, 463(ms) at org.apache.rocketmq.remoting.netty.NettyRemotingAbstract.invokeSyncImpl(NettyRemotingAbstract.java:391) at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:374) at org.apache.rocketmq.client.impl.MQClientAPIImpl.getBrokerClusterInfo(MQClientAPIImpl.java:1180) at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineBrokerClusterInfo(DefaultMQAdminExtImpl.java:275) at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.examineBrokerClusterInfo(DefaultMQAdminExt.java:222) at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.printClusterBaseInfo(ClusterListSubCommand.java:172) at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:88) ... 2 more [root@artisan bin]# ^C重试几次 [root@artisan bin]# ./mqadmin clusterlist -n 192.168.18.130:9876 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0 Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0 #Cluster Name#Broker Name#BID#Addr#Version#InTPS(LOAD)#OutTPS(LOAD) #PCWait(ms) #Hour #SPACE DefaultClusterbroker?a0192.168.18.130:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436999.54 0.1360 DefaultClusterbroker?a1192.168.18.130:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436999.54 0.1360 DefaultClusterbroker?b0192.168.18.131:10911V4_3_20.00(0,0ms)0.00(0,0ms)0 436999.54 0.1690 DefaultClusterbroker?b1192.168.18.131:10811V4_3_20.00(0,0ms)0.00(0,0ms)0 436999.54 0.1690 [root@artisan bin]#

可能是mqadmin没能正确的连接进去！网络抖动，这个跟底层netty的连接有关. 多重试几次，一般都没问题。
测试发送和消费消息

[root@artisan rocketmq-all-4.3.2-bin-release]# export NAMESRV_ADDR=192.168.18.131:9876 [root@artisan rocketmq-all-4.3.2-bin-release]# [root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/tools.shorg.apache.rocketmq.example.quickstart.Producer

结果截个图：