zookeeper集群安装小结 zookeeper集群安装小结

Zookeeper是一种在分布式系统中被广泛用来作为：分布式状态管理、分布式协调管理、分布式配置管理、和分布式锁服务的集群。kafka增加和减少服务器都会在Zookeeper节点上触发相应的事件kafka系统会捕获这些事件，进行新一轮的负载均衡，客户端也会捕获这些事件来进行新一轮的处理
安装过程可以参考
http://www.cnblogs.com/wrong5566/p/6056788.html
https://www.cnblogs.com/linuxprobe/p/5851699.html
要点：在三台服务器上配置zoo.cfg文件（来自zoo_sample.cfg文件的复制为zoo.cfg）

[root@zookeeper conf]# pwd /opt/zookeeper-3.4.12/conf [root@zookeeper conf]# vi zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/opt/zookeeper-3.4.12/data dataLogDir=/opt/zookeeper-3.4.12/logs # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 server.1= 192.168.197.129:2888:3888 server.2= 192.168.197.134:2888:3888 server.3= 192.168.197.136:2888:3888 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ "zoo.cfg" 34L, 1139C

【zookeeper集群安装小结】需要定义该目录：
dataDir=/opt/zookeeper-3.4.12/data
dataLogDir=/opt/zookeeper-3.4.12/logs
其次：
新建myid文件：
在三个linux机器上刚刚创建的data目录下新建一个myid文件并打开，vi myid命令。
在192.168.197.129机器上输入1，保存退出；在192.168.197.134机器上输入2，保存退出；在192.168.197.136机器上输入3，保存退出。这里的1,2,3是与server1,2,3相对应的。
启动：
需要先依次启动3台服务器上的zookeeper服务，
最后再查看状态有：leader和follower，证明启动成功
常用命令
[root@zk1 bin]# zkServer.sh start来启动。
zkServer.sh restart(重启)
zkServer.sh status(查看状态)
zkServer.sh stop(关闭)
zkServer.sh start-foreground(以打印日志方式启动)`
129上：

[root@xiehao1 bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@xiehao1 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: follower

134上：

[root@zookeeper bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@zookeeper bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: leader

136上：

[root@zookeeper bin]# ./zkServer.sh start ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@zookeeper bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: follower

**
zookeeper集群连接： **

Zookeeper集群搭建完毕之后，可以通过客户端脚本连接到zookeeper集群上面，对客户端来说，zookeeper集群是一个整体，连接到zookeeper集群实际上感觉在独享整个集群的服务。
在129上进行测试，连接到134上的zookeeper服务

[root@xiehao1 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Mode: follower [root@xiehao1 bin]# ./zkCli.sh -server 192.168.197.134:2181 Connecting to 192.168.197.134:2181 2019-04-28 23:47:50,484 [myid:] - INFO[main:Environment@100] - Client environment:zookeeper.version=3.4.12-e5259e437540f349646870ea94dc2658c4e44b3b, built on 03/27/2018 03:55 GMT 2019-04-28 23:47:50,497 [myid:] - INFO[main:Environment@100] - Client environment:host.name= 2019-04-28 23:47:50,497 [myid:] - INFO[main:Environment@100] - Client environment:java.version=1.8.0_161 2019-04-28 23:47:50,504 [myid:] - INFO[main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-28 23:47:50,504 [myid:] - INFO[main:Environment@100] - Client environment:java.home=/opt/jdk1.8.0_161/jre 2019-04-28 23:47:50,505 [myid:] - INFO[main:Environment@100] - Client environment:java.class.path=/opt/zookeeper-3.4.12/bin/../build/classes:/opt/zookeeper-3.4.12/bin/../build/lib/*.jar:/opt/zookeeper-3.4.12/bin/../lib/slf4j-log4j12-1.7.25.jar:/opt/zookeeper-3.4.12/bin/../lib/slf4j-api-1.7.25.jar:/opt/zookeeper-3.4.12/bin/../lib/netty-3.10.6.Final.jar:/opt/zookeeper-3.4.12/bin/../lib/log4j-1.2.17.jar:/opt/zookeeper-3.4.12/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.12/bin/../lib/audience-annotations-0.5.0.jar:/opt/zookeeper-3.4.12/bin/../zookeeper-3.4.12.jar:/opt/zookeeper-3.4.12/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.12/bin/../conf:.:/opt/jdk1.8.0_161/lib.tools.jar 2019-04-28 23:47:50,505 [myid:] - INFO[main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-28 23:47:50,505 [myid:] - INFO[main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-28 23:47:50,505 [myid:] - INFO[main:Environment@100] - Client environment:java.compiler= 2019-04-28 23:47:50,506 [myid:] - INFO[main:Environment@100] - Client environment:os.name=Linux 2019-04-28 23:47:50,506 [myid:] - INFO[main:Environment@100] - Client environment:os.arch=amd64 2019-04-28 23:47:50,506 [myid:] - INFO[main:Environment@100] - Client environment:os.version=2.6.32-642.el6.x86_64 2019-04-28 23:47:50,506 [myid:] - INFO[main:Environment@100] - Client environment:user.name=root 2019-04-28 23:47:50,506 [myid:] - INFO[main:Environment@100] - Client environment:user.home=/root 2019-04-28 23:47:50,507 [myid:] - INFO[main:Environment@100] - Client environment:user.dir=/opt/zookeeper-3.4.12/bin 2019-04-28 23:47:50,513 [myid:] - INFO[main:ZooKeeper@441] - Initiating client connection, connectString=192.168.197.134:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@4b85612c Welcome to ZooKeeper! 2019-04-28 23:47:50,610 [myid:] - INFO[main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@1028] - Opening socket connection to server 192.168.197.134/192.168.197.134:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-28 23:47:50,866 [myid:] - INFO[main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@878] - Socket connection established to 192.168.197.134/192.168.197.134:2181, initiating session 2019-04-28 23:47:51,027 [myid:] - INFO[main-SendThread(192.168.197.134:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server 192.168.197.134/192.168.197.134:2181, sessionid = 0x2000035a8500000, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null [zk: 192.168.197.134:2181(CONNECTED) 0] [zk: 192.168.197.134:2181(CONNECTED) 0] ls [zk: 192.168.197.134:2181(CONNECTED) 1] ls / [zookeeper] [zk: 192.168.197.134:2181(CONNECTED) 2] quit Quitting... 2019-04-28 23:48:41,983 [myid:] - INFO[main:ZooKeeper@687] - Session: 0x2000035a8500000 closed 2019-04-28 23:48:41,990 [myid:] - INFO[main-EventThread:ClientCnxn$EventThread@521] - EventThread shut down for session: 0x2000035a8500000 [root@xiehao1 bin]#

通过上图可以看出整个zookeeper集群已经搭建并测试完成。

文章图片

#!/bin/bash #snapshot file dir dataDir=/opt/zookeeper/zkdata/version-2 #tran log dir dataLogDir=/opt/zookeeper/zkdatalog/version-2#Leave 66 files count=66 count=$[$count+1] ls -t $dataLogDir/log.* | tail -n +$count | xargs rm -f ls -t $dataDir/snapshot.* | tail -n +$count | xargs rm -f #以上这个脚本定义了删除对应两个目录中的文件，保留最新的66个文件，可以将他写到crontab中，设置为每天凌晨2点执行一次就可以了。#zk log dirdel the zookeeper log #logDir= #ls -t $logDir/zookeeper.log.* | tail -n +$count | xargs rm -f

文章图片

文章图片
（图片博客链接https://www.cnblogs.com/luotianshuai/p/5206662.html）
错误记录：
当对三台机器同时 zkServer.sh start启动时，发现都启动成功了，但是以zkServer.sh status命令来查看启动状态时，发现总是在报错，如下图：

[root@xiehao1 bin]# ./zkServer.sh status ZooKeeper JMX enabled by default Using config: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg Error contacting service. It is probably not running.

查看报错日志（默认输出日志 bin/zookeeper.out）：

[root@xiehao1 zookeeper-3.4.12]# cd bin/ [root@xiehao1 bin]# ll total 40 -rwxrwxr--. 1 root root232 Mar 262018 README.txt -rwxrwxr--. 1 root root 1937 Mar 262018 zkCleanup.sh -rwxrwxr--. 1 root root 1056 Mar 262018 zkCli.cmd -rwxrwxr--. 1 root root 1534 Mar 262018 zkCli.sh -rwxrwxr--. 1 root root 1759 Mar 262018 zkEnv.cmd -rwxrwxr--. 1 root root 2696 Mar 262018 zkEnv.sh -rwxrwxr--. 1 root root 1089 Mar 262018 zkServer.cmd -rwxrwxr--. 1 root root 6773 Mar 262018 zkServer.sh -rw-r--r--. 1 root root 2090 Apr 28 20:09 zookeeper.out [root@xiehao1 bin]# cat zookeeper.out 2019-04-28 20:09:36,746 [myid:] - INFO[main:QuorumPeerConfig@136] - Reading configuration from: /opt/zookeeper-3.4.12/bin/../conf/zoo.cfg 2019-04-28 20:09:36,784 [myid:] - INFO[main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.136 to address: /192.168.197.136 2019-04-28 20:09:36,784 [myid:] - INFO[main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.134 to address: /192.168.197.134 2019-04-28 20:09:36,786 [myid:] - INFO[main:QuorumPeer$QuorumServer@184] - Resolved hostname: 192.168.197.129 to address: /192.168.197.129 2019-04-28 20:09:36,786 [myid:] - INFO[main:QuorumPeerConfig@398] - Defaulting to majority quorums 2019-04-28 20:09:36,794 [myid:1] - INFO[main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2019-04-28 20:09:36,794 [myid:1] - INFO[main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 0 2019-04-28 20:09:36,794 [myid:1] - INFO[main:DatadirCleanupManager@101] - Purge task is not scheduled. 2019-04-28 20:09:36,819 [myid:1] - INFO[main:QuorumPeerMain@130] - Starting quorum peer 2019-04-28 20:09:36,841 [myid:1] - INFO[main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory 2019-04-28 20:09:36,851 [myid:1] - INFO[main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 2019-04-28 20:09:36,893 [myid:1] - ERROR [main:QuorumPeerMain@92] - Unexpected exception, exiting abnormally org.apache.zookeeper.server.persistence.FileTxnSnapLog$SnapDirContentCheckException: Snapshot directory has log files. Check if dataLogDir and dataDir configuration is correct. at org.apache.zookeeper.server.persistence.FileTxnSnapLog.checkSnapDir(FileTxnSnapLog.java:135) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.(FileTxnSnapLog.java:108) at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:141) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)

网上说的常见的解决办法：
（1）：zoo.cfg配置文件中，server.x:2888:3888配置出现错误；
（2）：myid文件内容和server.x不对应，或者myid不在data目录下；
（3）：系统防火墙是否在启动。
经测试，没有以上问题，主要问题是：
（参考https://www.cnblogs.com/yinghao/p/6135095.html）
查看conf/zoo.cfg配置文件
dataDir 路径下删除 version-2文件夹
重新启动，解决问题！