Re: ZK based synchronization questions

2017-11-02 Thread Olivier Tavard
Hi Karl, OK it makes senses, thanks for the explanation ! Best regards, Olivier TAVARD > Le 31 oct. 2017 à 17:53, Karl Wright a écrit : > > Hi Olivier, > > Zookeeper connections are pooled, so they are pulled out of the pool at will > by ManifoldCF and returned when

Re: ZK based synchronization questions

2017-10-31 Thread Karl Wright
Hi Olivier, Zookeeper connections are pooled, so they are pulled out of the pool at will by ManifoldCF and returned when the lock etc is done. This means you really should need only a total number of outstanding Zookeeper handles that is on the same order as the number of operating threads in

Re: ZK based synchronization questions

2017-10-31 Thread Olivier Tavard
Hi all, Just to clarify my concern on ZK: To my knowledge, best practices concerning ZK connections are to not go beyond 60. Is there any rationale for setting it at 1000 for MCF ? Could this can have side effects on our ZK cluster shared by MCF and SolrCloud ? Thanks, Olivier > Le 23 oct.

ZK based synchronization questions

2017-10-23 Thread Olivier Tavard
Hello, We configured MCF to use ZK sync instead of file sync. We noticed a huge improvement regarding the stability of the MCF jobs in every case especially for large data to index (15M of files using the Windows Share repository connector). Before that, we had some errors when the job was