spring|Kafka ZooKeeper Len Error

1. Environment
Kafka 1.0, ZooKeeper 3.4.10, JDK 1.8 on Ubuntu 16.
Spring Cloud Edgware on Mac 10.13
2. Problem
启动spring cloud application. 连接zookeeper/kafka出错
spring cloud application error:

Caused by: org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 10000 at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1232) ~[zkclient-0.9.jar:na] at org.I0Itec.zkclient.ZkClient.(ZkClient.java:156) ~[zkclient-0.9.jar:na]



zookeeper server error: (Len Error)

[2017-12-11 14:03:46,563] INFO Accepted socket connection from /192.168.0.105:57805 (org.apache.zookeeper.server.NIOServerCnxnFactory) [2017-12-11 14:03:46,568] WARN Exception causing close of session 0x0 due to java.io.IOException: Len error 1397966893 (org.apache.zookeeper.server.NIOServerCnxn) [2017-12-11 14:03:46,569] INFO Closed socket connection for client /192.168.0.105:57805 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)


3. Analysis
【spring|Kafka ZooKeeper Len Error】http://blog.csdn.net/liuxinghao/article/details/59071373

https://community.hortonworks.com/content/supportkb/150660/error-javaioioexception-len-error-in-zookeeper-cau.html

This issue occurs when the amount of data being stored or retrieved on the znode exceeds the jute.maxbuffer. For example, If a Zookeeper client (Kafka broker) sets a large number of watches, the "set watches" operation during session re-establishment can fail.

Kafka stores all the metadata information of topics and partitions in the Zookeeper in Znodes. To resolve this issue, increase jute.maxbuffer on the Kafka brokers and Zookeeper server. To increase jute.maxbuffer, add -Djute.maxbuffer to 20000000 in JAVA OPTS for Kafka and Zookeeper.



official doc: jute.maxbuffer: (Java system property: jute.maxbuffer)
This option can only be set as a Java system property. There is no zookeeper prefix on it. It specifies the maximum size of the data that can be stored in a znode. The default is 0xfffff, or just under 1M. If this option is changed, the system propertymust be set on all servers and clients otherwise problems will arise. This is really a sanity check. ZooKeeper is designed to store data on the order of kilobytes in size.



Java Application Client Configuration: -Djute.maxbuffer=41943040
zkCli.sh Configuration: -Djute.maxbuffer=41943040
zkServer.sh Configuration:-Djute.maxbuffer=41943040(reference: http://blog.csdn.net/xfg0218/article/details/52709917)




    推荐阅读