Re: Cluster behavior when heavily loaded

2018-05-09 Thread Jeff
Mark, You could try increasing the "tickTime" setting in zookeeper.properties to give ZK a little more time before timing something out. Ticktime is defined [1] as "the length of a single tick, which is the basic time unit used by ZooKeeper, as measured in milliseconds. It is used to regulate

Cluster behavior when heavily loaded

2018-05-08 Thread Mark Bean
We have a 6-node cluster using external ZooKeeper. It is heavily loaded, and we are attempting to tune some of the properties to alleviate some observed issues. By "heavily loaded" I mean the graph is large (approx. 3,000 processors) and there is a lot of data in process (approx. 2M