Hi all, Just to clarify my concern on ZK: To my knowledge, best practices concerning ZK connections are to not go beyond 60. Is there any rationale for setting it at 1000 for MCF ? Could this can have side effects on our ZK cluster shared by MCF and SolrCloud ?
Thanks, Olivier > Le 23 oct. 2017 à 17:19, Olivier Tavard <olivier.tav...@francelabs.com> a > écrit : > > Hello, > > We configured MCF to use ZK sync instead of file sync. We noticed a huge > improvement regarding the stability of the MCF jobs in every case especially > for large data to index (15M of files using the Windows Share repository > connector). Before that, we had some errors when the job was running > randomly. With that change, we did not notice any error on the job so far. > However, after testing that configuration on several servers, we had errors > reported and I would like to know what you suggest for that. > We installed MCF on servers that already have Solr 6.6.X on them. I saw on > other threads on the mailing list that it was OK to use existing ZK > installation rather than using a new ZK instance dedicated to MCF so we use > the same ZK for both Solr and MCF. > After starting MCF and Solr, we noticed some errors on the MCF log for few > servers : Session 0x0 for server localhost/127.0.0.1:2181, unexpected error, > closing socket connection and attempting reconnect > java.io.IOException: Connection reset by peer > Then after checking the ZK log we saw this message : "WARN 2017-10-23 > 08:53:35,431 (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181) - > ZooKeeper|ZooKeeper|zookeeper.server.NIOServerCnxnFactory|Too many > connections from /127.0.0.1 - max is 60" > Therefore we changed the parameter maxClientCnxns from 60 in our ZK > configuration to 1000 as in the MCF zookeeper.cfg default file and there is > no problem anymore. > I would just like to know why this parameter needs to be so high for MCF > needs and if other people share their ZK cluster with both Solr and MCF too > without any problem. And last question I had : MCF uses Zookeeper 3.4.8 while > Solr 6.6+ uses ZK 3.4.10. As written above our ZK cluster version is 3.4.10 > and we use MCF on it, is it ok to do that or would it be best to use a ZK > installation with version 3.4.8 only for MCF ? So far we did not see any > problems using it. > > Thanks, > Best regards, > > Olivier TAVARD
