Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-08-14 Thread Andor Molnar
After some digging it turned out that this is an outstanding issue in 3.4->3.5 upgrade. I’ve found the following e-mail thread about it: https://markmail.org/thread/rbhzbro6nszypwwp …and an open Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-3056 Unfortunately, patch is still not availabl

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-08-14 Thread Andor Molnar
Hi Jorn, Thanks for reaching out to us, this is a very important exercise to make sure the upgrade path works as expected. - Please do an `ls -al` in your data dir to make sure you have valid snapshot files. - It would be also useful to expose the Admin port (8080/tcp by default) and check the

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-08-13 Thread Jörn Franke
For me the issue occurred only in standalone mode. With the ensemble I simply cleared the data directory and it received the zookeeper data from the quorum. > Am 13.08.2019 um 15:42 schrieb Koen De Groote : > > I would also like to know if this is possible. > > From going over the github page,

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-08-13 Thread Enrico Olivelli
Il mar 13 ago 2019, 15:43 Koen De Groote ha scritto: > I would also like to know if this is possible. > > From going over the github page, it seems there is a JMX method to force > the creation of a snapshot. Yet the docker image is configured as such that > a port will never be assigned to the J

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-08-13 Thread Koen De Groote
I would also like to know if this is possible. >From going over the github page, it seems there is a JMX method to force the creation of a snapshot. Yet the docker image is configured as such that a port will never be assigned to the JMX process. Is there any way to bypass this? On Tue, Jul 30,

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
Thanks. It is possible to force Zookeeper to create a snapshot? I will check I think the snapshot count is set to 1 in the cfg > Am 30.07.2019 um 08:06 schrieb Enrico Olivelli : > > Il giorno lun 29 lug 2019 alle ore 23:59 Jörn Franke > ha scritto: > >> ok, then let me verify tomorrow if a sn

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Enrico Olivelli
Il giorno lun 29 lug 2019 alle ore 23:59 Jörn Franke ha scritto: > ok, then let me verify tomorrow if a snapshot file is indeed there. If it > is missing then I wonder why it was missing. There was no crash or whatever > and 3.4.14 works without issue, but of course it could have loaded them > fr

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
ok, then let me verify tomorrow if a snapshot file is indeed there. If it is missing then I wonder why it was missing. There was no crash or whatever and 3.4.14 works without issue, but of course it could have loaded them from the log files. However, then I wonder why it does not create one. On Mo

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Michael Han
>> I just wonder why it does not find a valid snapshot. If there are local snapshot files and the files are valid, then it's a bug that server fails to load them. >> Is it because the format changed in 3.5.5 compared to 3.4.14? Not I am aware of. There are some format changes (added compression

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
ok, then it affects basically all standalone nodes? This is fine, despite that it means some extra work (for uncritical lab environments). I am not sure it is ZOOKEEPER-2325, but I don't know the full history behind it).The logs are fine (it works in 3.4.14 without issues, even after downgrading ba

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Michael Han
>> java.io.IOException: No snapshot found, but there are log entries. Something is broken! This is expected behavior introduced in ZOOKEEPER-2325. We don't want to end up with potential inconsistent state across the ensemble when recovering from empty snapshot. To continue upgrade, just delete al

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Enrico Olivelli
Il lun 29 lug 2019, 22:32 Jörn Franke ha scritto: > It also seems that 3.5.5 does not attempt to read all of the logfiles (I > have to still confirm), but the two it reads exist, it has access and they > are much more than 0 byte > We should have the stackstace of the EOFException. Anyone on th

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
It also seems that 3.5.5 does not attempt to read all of the logfiles (I have to still confirm), but the two it reads exist, it has access and they are much more than 0 byte On Mon, Jul 29, 2019 at 10:13 PM Jörn Franke wrote: > (of course i do not run them at the same time) > > On Mon, Jul 29, 2

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
(of course i do not run them at the same time) On Mon, Jul 29, 2019 at 10:10 PM Jörn Franke wrote: > thank you for the quick reply. They read from the same disk paths and have > the same access rights (in fact the RHEL service executes them as the same > specific user). > > On Mon, Jul 29, 2019

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
thank you for the quick reply. They read from the same disk paths and have the same access rights (in fact the RHEL service executes them as the same specific user). On Mon, Jul 29, 2019 at 10:09 PM Enrico Olivelli wrote: > Il lun 29 lug 2019, 21:50 Jörn Franke ha scritto: > > > Hi, > > > > I t

Re: Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Enrico Olivelli
Il lun 29 lug 2019, 21:50 Jörn Franke ha scritto: > Hi, > > I tried to migrate a lab environment from Zookeepr 3.4.14 (used for Solr) > to 3.5.5 and encountered an issue. It is ZooKeeper in standalone mode > (other environments have a proper ensemble). I increased jute.maxbuffer > beyond the defa

Issue migrating from Zookeeper 3.4.14 to 3.5.5

2019-07-29 Thread Jörn Franke
Hi, I tried to migrate a lab environment from Zookeepr 3.4.14 (used for Solr) to 3.5.5 and encountered an issue. It is ZooKeeper in standalone mode (other environments have a proper ensemble). I increased jute.maxbuffer beyond the default (but not excessively) - this was working perfectly fine in