Re: Getting this error in ZK server logs : Unexpected exception causing shutdown while sock still open java.io.EOFException
Already posted about that a few days ago. still awaiting for an answer. We are using 3.5.3-beta On 19/07/17 11:36, prashantkumar dhotre wrote: Hi I am seeing below error in ZK logs : "Unexpected exception causing shutdown while sock still open java.io.EOFException". I am using zookeeper-3.5.1-alpha version. Could you please help me resolve this issue Thanks 114829 2017-06-22 11:24:18,182 [myid:2147483652] - INFO [ProcessThread(sid:2147483652 cport:-1)::PrepRequestProcessor@649] - Processed session termination for sessionid: 0x4007cef003d 114830 2017-06-22 11:24:18,182 [myid:2147483652] - INFO [NIOWorkerThread-8:MBeanRegistry@119] - Unregister MBean [org.apache.ZooKeeperService:name0=ReplicatedServer_id2147483652, name1=replica.2147483652,name2=Leader,name3=Connections,name4=128.0.0.5,name5=0x4007cef003d] 114831 2017-06-22 11:24:18,183 [myid:2147483652] - INFO [NIOWorkerThread-8:NIOServerCnxn@606] - Closed socket connection for client /128.0.0.5:34651 which had sessionid 0x4007cef003d 114832 2017-06-22 11:24:18,421 [myid:2147483652] - ERROR [LearnerHandler-/128.0.0.5:33610:LearnerHandler@604] - Unexpected exception causing shutdown while sock still open 114833 java.io.EOFException 114834 at java.io.DataInputStream.readInt(DataInputStream.java:403) 114835 at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) 114836 at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) 114837 at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:99) 114838 at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:515) 114839 2017-06-22 11:24:18,422 [myid:2147483652] - WARN [LearnerHandler-/128.0.0.5:33610:LearnerHandler@619] - *** GOODBYE / 128.0.0.5:33610 114840 2017-06-22 11:24:18,422 [myid:2147483652] - INFO [NIOServerCxnFactory.AcceptThread:/0.0.0.0:61808 :NIOServerCnxnFactory$AcceptThread@296] - Accepted socket connection from / 128.0.0.4:42854 114841 2017-06-22 11:24:18,422 [myid:2147483652] - INFO [NIOWorkerThread-4:ZooKeeperServer@969] - Client attempting to renew session 0x4007cef0001 at /128.0.0.4:42854 114842 2017-06-22 11:24:18,422 [myid:2147483652] - INFO [NIOWorkerThread-4:ZooKeeperServer@678] - Established session 0x 4007cef0001 with negotiated timeout 2 for client / 128.0.0.4:42854 114843 2017-06-22 11:24:18,423 [myid:2147483652] - INFO [NIOServerCxnFactory.AcceptThread:/0.0.0.0:61808 :NIOServerCnxnFactory$AcceptThread@296] - Accepted socket connection from / 128.0.0.4:42862 114844 2017-06-22 11:24:18,423 [myid:2147483652] - INFO [NIOServerCxnFactory.AcceptThread:/0.0.0.0:61808:NIOSe + After some time .. 114851 2017-06-22 11:24:18,423 [myid:2147483652] - INFO [NIOWorkerThread-12:ZooKeeperServer@678] - Established session 0x4007cef0003 with negotiated timeout 2 for client / 128.0.0.4:42866 114852 2017-06-22 11:24:19,001 [myid:2147483652] - INFO [NIOServerCxnFactory.AcceptThread:/0.0.0.0:61808 :NIOServerCnxnFactory$AcceptThread@296] - Accepted socket connection from / 128.0.0.4:42892 114853 2017-06-22 11:24:19,001 [myid:2147483652] - INFO [NIOWorkerThread-13:ZooKeeperServer@964] - Client attempting to establish new session at /128.0.0.4:42892 114854 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [SessionTracker:ZooKeeperServer@384] - Expiring session 0x4007cef016c, timeout of 2ms exceeded 114855 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [SessionTracker:QuorumZooKeeperServer@132] - Submitting global closeSession request for session 0x4007cef016c 114856 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [SessionTracker:ZooKeeperServer@384] - Expiring session 0x4007cef016d, timeout of 2ms exceeded 114857 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [ProcessThread(sid:2147483652 cport:-1)::PrepRequestProcessor@649] - Processed session termination for sessionid: 0x4007cef016c 114858 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [SessionTracker:QuorumZooKeeperServer@132] - Submitting global closeSession request for session 0x4007cef016d 114859 2017-06-22 11:24:19,211 [myid:2147483652] - INFO [ProcessThread(sid:2147483652 cport:-1)::PrepRequestProcessor@649] - Processed session termination for sessionid: 0x4007cef016d 114860 2017-06-22 11:24:19,579 [myid:2147483652] - INFO [QuorumPeer[myid=2147483652](plain=/0:0:0:0:0:0:0:0:61808)(secure=disabled):Leader@613] - Shutting down 114861 2017-06-22 11:24:19,579 [myid:2147483652] - INFO [QuorumPeer[myid=2147483652](plain=/0:0:0:0:0:0:0:0:61808)(secure=disabled):Leader@619] - Shutdown called 114862 java.lang.Exception: shutdown Leader! reason: Not sufficient followers synced, only synced with sids: [ [2147483652] ] 114863 at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:619) 114864 at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:59
RE: java.io.EOFException
You need to send mail to user-unsubscr...@zookeeper.apache.org<mailto:user-unsubscr...@zookeeper.apache.org>. Reference: https://zookeeper.apache.org/lists.html Brahma Reddy Battula From: Mike Richardson [mailto:m...@motum.be] Sent: 30 June 2017 14:38 To: user@zookeeper.apache.org Subject: Re: java.io.EOFException Unsubscribe Mike Richardson Senior Software Engineer [cid:part1.BC028947.AB80697F@motum.be] MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium T +32(0)15 28 16 63 M +32 494 63 98 15 www.motum.be<http://www.motum.be/> On 29 June 2017 at 18:59, Michael Han <h...@cloudera.com<mailto:h...@cloudera.com>> wrote: On Wed, Jun 28, 2017 at 11:59 PM, Mike Richardson <m...@motum.be<mailto:m...@motum.be>> wrote: > Unsubscribe > > > Unsubscribe does not work like this. To unsubscribe, please click the Unsubscribe from List <%75%73%65%72%2D%75%6E%73%75%62%73%63%72%69%62%65%40%7A%6F%6F%6B%65%65%70%65%72%2E%61%70%61%63%68%65%2E%6F%72%67> link from https://zookeeper.apache.org/lists.html. > Mike Richardson > > Senior Software Engineer > > > > *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* > > > T +32(0)15 28 16 63 <+32%2015%2028%2016%2063> > M +32 494 63 98 15 <+32%20494%2063%2098%2015> > > > www.motum.be<http://www.motum.be> > > On 28 June 2017 at 18:12, upendar devu > <devulapal...@gmail.com<mailto:devulapal...@gmail.com>> wrote: > >> We have 3 instances of zookeeper and seen below error and then the >> zookeeper server process was auto stopped for a while and then it was auto >> restarted. >> >> We see this error at least once in a month . we have data directory >> within >> the zookeeper AWS instance could you please help us. >> >> Thanks >> >> Zookeeper log : >> >> >> 2017-06-27 21:24:58,839 [myid:3] - WARN >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of stream >> exception >> EndOfStreamException: Unable to read additional data from client sessionid >> 0x0, likely client has closed socket >> at >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) >> at >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServ >> erCnxnFactory.java:208) >> at java.lang.Thread.run(Thread.java:745) >> 2017-06-27 21:24:58,839 [myid:3] - INFO >> [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed >> socket connection for client /XX.XX.XX.XX:57497 (no sessi >> on established for client) >> 2017-06-27 21:24:58,840 [myid:3] - INFO [ >> zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumC<http://zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumC> >> nxManager$Listener@511] >> - Received connection request /XX.XX>XX >> 2:35419 >> 2017-06-27 21:24:58,840 [myid:3] - WARN [ >> zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260<http://zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260>] >> - >> Exception reading or writing challenge: java.io.E >> OFException >> 2017-06-27 21:24:58,840 [myid:3] - ERROR >> [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception >> causing shutdown while sock still open >> java.io.EOFException >> at java.io.DataInputStream.readInt(DataInputStream.java:392) >> at >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) >> at >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize( >> QuorumPacket.java:83) >> at >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArc >> hive.java:103) >> at >> org.apache.zookeeper.server.quorum.LearnerHandler.run(Learne >> rHandler.java:309) >> 2017-06-27 21:24:58,841 [myid:3] - WARN >> [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE >> /:39920 >> 2017-06-27 21:25:03,187 [myid:3] - INFO >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxnFactory@197] - Accepted >> socket >> connection from /1XXX:36060 >> 2017-06-27 21:25:03,187 [myid:3] - INFO >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@827] - Processing srvr >> command from /XXX:36060 >> 2017-06-27 21:25:03,188 [myid:3] - INFO [Thread-661253:NIOServerCnxn@1 >> 007] >> - Closed socket connection for client /XXX:36060 (no session establis >> 2017-06-27 21:27:17,496 [myid:3] - INFO >> [CommitProcessor:3:ZooKeeperServer@617] - Established session >> 0x35b125b84aa028a with negotiated timeout 3 for client /IP:40526 >> 2017-06-27 21:27:38,763 [myid:] - I
Re: java.io.EOFException
Unsubscribe Mike Richardson Senior Software Engineer *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* T +32(0)15 28 16 63 M +32 494 63 98 15 www.motum.be On 29 June 2017 at 18:59, Michael Han <h...@cloudera.com> wrote: > On Wed, Jun 28, 2017 at 11:59 PM, Mike Richardson <m...@motum.be> wrote: > > > Unsubscribe > > > > > > > Unsubscribe does not work like this. To unsubscribe, please click the > Unsubscribe > from List > <%75%73%65%72%2D%75%6E%73%75%62%73%63%72%69%62%65%40%7A%6F% > 6F%6B%65%65%70%65%72%2E%61%70%61%63%68%65%2E%6F%72%67> > link > from https://zookeeper.apache.org/lists.html. > > > > Mike Richardson > > > > Senior Software Engineer > > > > > > > > *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* > > > > > > T +32(0)15 28 16 63 <+32%2015%2028%2016%2063> > > M +32 494 63 98 15 <+32%20494%2063%2098%2015> > > > > > > www.motum.be > > > > On 28 June 2017 at 18:12, upendar devu <devulapal...@gmail.com> wrote: > > > >> We have 3 instances of zookeeper and seen below error and then the > >> zookeeper server process was auto stopped for a while and then it was > auto > >> restarted. > >> > >> We see this error at least once in a month . we have data directory > >> within > >> the zookeeper AWS instance could you please help us. > >> > >> Thanks > >> > >> Zookeeper log : > >> > >> > >> 2017-06-27 21:24:58,839 [myid:3] - WARN > >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of > stream > >> exception > >> EndOfStreamException: Unable to read additional data from client > sessionid > >> 0x0, likely client has closed socket > >> at > >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > >> at > >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServ > >> erCnxnFactory.java:208) > >> at java.lang.Thread.run(Thread.java:745) > >> 2017-06-27 21:24:58,839 [myid:3] - INFO > >> [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed > >> socket connection for client /XX.XX.XX.XX:57497 (no sessi > >> on established for client) > >> 2017-06-27 21:24:58,840 [myid:3] - INFO [ > >> zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumC > >> nxManager$Listener@511] > >> - Received connection request /XX.XX>XX > >> 2:35419 > >> 2017-06-27 21:24:58,840 [myid:3] - WARN [ > >> zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - > >> Exception reading or writing challenge: java.io.E > >> OFException > >> 2017-06-27 21:24:58,840 [myid:3] - ERROR > >> [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception > >> causing shutdown while sock still open > >> java.io.EOFException > >> at java.io.DataInputStream.readInt(DataInputStream.java:392) > >> at > >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > >> at > >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize( > >> QuorumPacket.java:83) > >> at > >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArc > >> hive.java:103) > >> at > >> org.apache.zookeeper.server.quorum.LearnerHandler.run(Learne > >> rHandler.java:309) > >> 2017-06-27 21:24:58,841 [myid:3] - WARN > >> [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE > >> /:39920 > >> 2017-06-27 21:25:03,187 [myid:3] - INFO > >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxnFactory@197] - Accepted > >> socket > >> connection from /1XXX:36060 > >> 2017-06-27 21:25:03,187 [myid:3] - INFO > >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@827] - Processing srvr > >> command from /XXX:36060 > >> 2017-06-27 21:25:03,188 [myid:3] - INFO [Thread-661253:NIOServerCnxn@1 > >> 007] > >> - Closed socket connection for client /XXX:36060 (no session establis > >> 2017-06-27 21:27:17,496 [myid:3] - INFO > >> [CommitProcessor:3:ZooKeeperServer@617] - Established session > >> 0x35b125b84aa028a with negotiated timeout 3 for client /IP:40526 > >> 2017-06-27 21:27:38,763 [myid:] - INFO [main:QuorumPeerConfig@103] - > >> Reading configuration from: /opt/zookeeper/current/bin/../conf/zoo.cfg > >> 2017-06-27 21:27:38,780 [myid:] - INFO [main:QuorumPeerConfig@340] - > >> Defaulting to majority quorums > >> 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@78 > ] > >> - > >> autopurge.snapRetainCount set to 3 > >> 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@79 > ] > >> - > >> autopurge.purgeInterval set to 1 > >> 2017-06-27 21:27:38,799 [myid:3] - INFO > >> [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started. > >> 2017-06-27 21:27:38,821 [myid:3] - INFO [main:QuorumPeerMain@127] - > >> Starting quorum peer > >> 2017-06-27 21:27:38,869 [myid:3] - INFO [main:NIOServerCnxnFactory@94] > - > >> binding to port /IP:2181 > >> 2017-06-27 21:27:38, > >> > > > > > > > -- > Cheers > Michael. >
Re: java.io.EOFException
Hi Upendar- The exception that is most concerning is: 2017-06-27 21:24:58,840 [myid:3] - WARN [ zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - Exception reading or writing challenge: java.io.E OFException 2017-06-27 21:24:58,840 [myid:3] - ERROR [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception causing shutdown while sock still open java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:309) In my experience this is related to timeouts that are too small on your learners (the timeouts here are derived from the tickTime, syncLimit, and initLimit)? Are you able to send the corresponding exception that occurred on the learner (that may make what's going on here a little clearer)? Thanks, Abe On Thu, Jun 29, 2017, at 09:59, Michael Han wrote: > On Wed, Jun 28, 2017 at 11:59 PM, Mike Richardson <m...@motum.be> wrote: > > > Unsubscribe > > > > > > > Unsubscribe does not work like this. To unsubscribe, please click the > Unsubscribe > from List > <%75%73%65%72%2D%75%6E%73%75%62%73%63%72%69%62%65%40%7A%6F%6F%6B%65%65%70%65%72%2E%61%70%61%63%68%65%2E%6F%72%67> > link > from https://zookeeper.apache.org/lists.html. > > > > Mike Richardson > > > > Senior Software Engineer > > > > > > > > *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* > > > > > > T +32(0)15 28 16 63 <+32%2015%2028%2016%2063> > > M +32 494 63 98 15 <+32%20494%2063%2098%2015> > > > > > > www.motum.be > > > > On 28 June 2017 at 18:12, upendar devu <devulapal...@gmail.com> wrote: > > > >> We have 3 instances of zookeeper and seen below error and then the > >> zookeeper server process was auto stopped for a while and then it was auto > >> restarted. > >> > >> We see this error at least once in a month . we have data directory > >> within > >> the zookeeper AWS instance could you please help us. > >> > >> Thanks > >> > >> Zookeeper log : > >> > >> > >> 2017-06-27 21:24:58,839 [myid:3] - WARN > >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of stream > >> exception > >> EndOfStreamException: Unable to read additional data from client sessionid > >> 0x0, likely client has closed socket > >> at > >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > >> at > >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServ > >> erCnxnFactory.java:208) > >> at java.lang.Thread.run(Thread.java:745) > >> 2017-06-27 21:24:58,839 [myid:3] - INFO > >> [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed > >> socket connection for client /XX.XX.XX.XX:57497 (no sessi > >> on established for client) > >> 2017-06-27 21:24:58,840 [myid:3] - INFO [ > >> zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumC > >> nxManager$Listener@511] > >> - Received connection request /XX.XX>XX > >> 2:35419 > >> 2017-06-27 21:24:58,840 [myid:3] - WARN [ > >> zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - > >> Exception reading or writing challenge: java.io.E > >> OFException > >> 2017-06-27 21:24:58,840 [myid:3] - ERROR > >> [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception > >> causing shutdown while sock still open > >> java.io.EOFException > >> at java.io.DataInputStream.readInt(DataInputStream.java:392) > >> at > >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > >> at > >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize( > >> QuorumPacket.java:83) > >> at > >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArc > >> hive.java:103) > >> at > >> org.apache.zookeeper.server.quorum.LearnerHandler.run(Learne > >> rHandler.java:309) > >> 2017-06-27 21:24:58,841 [myid:3] - WARN > >> [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE > >> /:39920 > >> 2017-06-27 21:25:03,187 [myid:3] - INFO > >> [NIOServerC
Re: java.io.EOFException
On Wed, Jun 28, 2017 at 11:59 PM, Mike Richardson <m...@motum.be> wrote: > Unsubscribe > > > Unsubscribe does not work like this. To unsubscribe, please click the Unsubscribe from List <%75%73%65%72%2D%75%6E%73%75%62%73%63%72%69%62%65%40%7A%6F%6F%6B%65%65%70%65%72%2E%61%70%61%63%68%65%2E%6F%72%67> link from https://zookeeper.apache.org/lists.html. > Mike Richardson > > Senior Software Engineer > > > > *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* > > > T +32(0)15 28 16 63 <+32%2015%2028%2016%2063> > M +32 494 63 98 15 <+32%20494%2063%2098%2015> > > > www.motum.be > > On 28 June 2017 at 18:12, upendar devu <devulapal...@gmail.com> wrote: > >> We have 3 instances of zookeeper and seen below error and then the >> zookeeper server process was auto stopped for a while and then it was auto >> restarted. >> >> We see this error at least once in a month . we have data directory >> within >> the zookeeper AWS instance could you please help us. >> >> Thanks >> >> Zookeeper log : >> >> >> 2017-06-27 21:24:58,839 [myid:3] - WARN >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of stream >> exception >> EndOfStreamException: Unable to read additional data from client sessionid >> 0x0, likely client has closed socket >> at >> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) >> at >> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServ >> erCnxnFactory.java:208) >> at java.lang.Thread.run(Thread.java:745) >> 2017-06-27 21:24:58,839 [myid:3] - INFO >> [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed >> socket connection for client /XX.XX.XX.XX:57497 (no sessi >> on established for client) >> 2017-06-27 21:24:58,840 [myid:3] - INFO [ >> zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumC >> nxManager$Listener@511] >> - Received connection request /XX.XX>XX >> 2:35419 >> 2017-06-27 21:24:58,840 [myid:3] - WARN [ >> zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - >> Exception reading or writing challenge: java.io.E >> OFException >> 2017-06-27 21:24:58,840 [myid:3] - ERROR >> [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception >> causing shutdown while sock still open >> java.io.EOFException >> at java.io.DataInputStream.readInt(DataInputStream.java:392) >> at >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) >> at >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize( >> QuorumPacket.java:83) >> at >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArc >> hive.java:103) >> at >> org.apache.zookeeper.server.quorum.LearnerHandler.run(Learne >> rHandler.java:309) >> 2017-06-27 21:24:58,841 [myid:3] - WARN >> [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE >> /:39920 >> 2017-06-27 21:25:03,187 [myid:3] - INFO >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxnFactory@197] - Accepted >> socket >> connection from /1XXX:36060 >> 2017-06-27 21:25:03,187 [myid:3] - INFO >> [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@827] - Processing srvr >> command from /XXX:36060 >> 2017-06-27 21:25:03,188 [myid:3] - INFO [Thread-661253:NIOServerCnxn@1 >> 007] >> - Closed socket connection for client /XXX:36060 (no session establis >> 2017-06-27 21:27:17,496 [myid:3] - INFO >> [CommitProcessor:3:ZooKeeperServer@617] - Established session >> 0x35b125b84aa028a with negotiated timeout 3 for client /IP:40526 >> 2017-06-27 21:27:38,763 [myid:] - INFO [main:QuorumPeerConfig@103] - >> Reading configuration from: /opt/zookeeper/current/bin/../conf/zoo.cfg >> 2017-06-27 21:27:38,780 [myid:] - INFO [main:QuorumPeerConfig@340] - >> Defaulting to majority quorums >> 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@78] >> - >> autopurge.snapRetainCount set to 3 >> 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@79] >> - >> autopurge.purgeInterval set to 1 >> 2017-06-27 21:27:38,799 [myid:3] - INFO >> [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started. >> 2017-06-27 21:27:38,821 [myid:3] - INFO [main:QuorumPeerMain@127] - >> Starting quorum peer >> 2017-06-27 21:27:38,869 [myid:3] - INFO [main:NIOServerCnxnFactory@94] - >> binding to port /IP:2181 >> 2017-06-27 21:27:38, >> > > -- Cheers Michael.
Re: java.io.EOFException
Unsubscribe Mike Richardson Senior Software Engineer *MoTuM N.V. | Dellingstraat 34 | B-2800 MECHELEN | Belgium* T +32(0)15 28 16 63 M +32 494 63 98 15 www.motum.be On 28 June 2017 at 18:12, upendar devu <devulapal...@gmail.com> wrote: > We have 3 instances of zookeeper and seen below error and then the > zookeeper server process was auto stopped for a while and then it was auto > restarted. > > We see this error at least once in a month . we have data directory within > the zookeeper AWS instance could you please help us. > > Thanks > > Zookeeper log : > > > 2017-06-27 21:24:58,839 [myid:3] - WARN > [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of stream > exception > EndOfStreamException: Unable to read additional data from client sessionid > 0x0, likely client has closed socket > at > org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.run( > NIOServerCnxnFactory.java:208) > at java.lang.Thread.run(Thread.java:745) > 2017-06-27 21:24:58,839 [myid:3] - INFO > [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed > socket connection for client /XX.XX.XX.XX:57497 (no sessi > on established for client) > 2017-06-27 21:24:58,840 [myid:3] - INFO [ > zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888: > QuorumCnxManager$Listener@511] > - Received connection request /XX.XX>XX > 2:35419 > 2017-06-27 21:24:58,840 [myid:3] - WARN [ > zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - > Exception reading or writing challenge: java.io.E > OFException > 2017-06-27 21:24:58,840 [myid:3] - ERROR > [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception > causing shutdown while sock still open > java.io.EOFException > at java.io.DataInputStream.readInt(DataInputStream.java:392) > at > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > at > org.apache.zookeeper.server.quorum.QuorumPacket. > deserialize(QuorumPacket.java:83) > at > org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) > at > org.apache.zookeeper.server.quorum.LearnerHandler.run( > LearnerHandler.java:309) > 2017-06-27 21:24:58,841 [myid:3] - WARN > [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE > /:39920 > 2017-06-27 21:25:03,187 [myid:3] - INFO > [NIOServerCxn.Factory:/IP:2181:NIOServerCnxnFactory@197] - Accepted > socket > connection from /1XXX:36060 > 2017-06-27 21:25:03,187 [myid:3] - INFO > [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@827] - Processing srvr > command from /XXX:36060 > 2017-06-27 21:25:03,188 [myid:3] - INFO [Thread-661253:NIOServerCnxn@ > 1007] > - Closed socket connection for client /XXX:36060 (no session establis > 2017-06-27 21:27:17,496 [myid:3] - INFO > [CommitProcessor:3:ZooKeeperServer@617] - Established session > 0x35b125b84aa028a with negotiated timeout 3 for client /IP:40526 > 2017-06-27 21:27:38,763 [myid:] - INFO [main:QuorumPeerConfig@103] - > Reading configuration from: /opt/zookeeper/current/bin/../conf/zoo.cfg > 2017-06-27 21:27:38,780 [myid:] - INFO [main:QuorumPeerConfig@340] - > Defaulting to majority quorums > 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@78] - > autopurge.snapRetainCount set to 3 > 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@79] - > autopurge.purgeInterval set to 1 > 2017-06-27 21:27:38,799 [myid:3] - INFO > [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started. > 2017-06-27 21:27:38,821 [myid:3] - INFO [main:QuorumPeerMain@127] - > Starting quorum peer > 2017-06-27 21:27:38,869 [myid:3] - INFO [main:NIOServerCnxnFactory@94] - > binding to port /IP:2181 > 2017-06-27 21:27:38, >
java.io.EOFException
We have 3 instances of zookeeper and seen below error and then the zookeeper server process was auto stopped for a while and then it was auto restarted. We see this error at least once in a month . we have data directory within the zookeeper AWS instance could you please help us. Thanks Zookeeper log : 2017-06-27 21:24:58,839 [myid:3] - WARN [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@357] - caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely client has closed socket at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) at java.lang.Thread.run(Thread.java:745) 2017-06-27 21:24:58,839 [myid:3] - INFO [NIOServerCxn.Factory:/1XX.XX.XX.XX:2181:NIOServerCnxn@1007] - Closed socket connection for client /XX.XX.XX.XX:57497 (no sessi on established for client) 2017-06-27 21:24:58,840 [myid:3] - INFO [ zookeeper3-prod.aws.npp.neustar.biz/XX.XX.XX.XX:3888:QuorumCnxManager$Listener@511] - Received connection request /XX.XX>XX 2:35419 2017-06-27 21:24:58,840 [myid:3] - WARN [ zookeeper3-prod.aws.npp.neustar.biz/:3888:QuorumCnxManager@260] - Exception reading or writing challenge: java.io.E OFException 2017-06-27 21:24:58,840 [myid:3] - ERROR [LearnerHandler-/:39920:LearnerHandler@633] - Unexpected exception causing shutdown while sock still open java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) at org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:103) at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:309) 2017-06-27 21:24:58,841 [myid:3] - WARN [LearnerHandler-/IP:39920:LearnerHandler@646] - *** GOODBYE /:39920 2017-06-27 21:25:03,187 [myid:3] - INFO [NIOServerCxn.Factory:/IP:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /1XXX:36060 2017-06-27 21:25:03,187 [myid:3] - INFO [NIOServerCxn.Factory:/IP:2181:NIOServerCnxn@827] - Processing srvr command from /XXX:36060 2017-06-27 21:25:03,188 [myid:3] - INFO [Thread-661253:NIOServerCnxn@1007] - Closed socket connection for client /XXX:36060 (no session establis 2017-06-27 21:27:17,496 [myid:3] - INFO [CommitProcessor:3:ZooKeeperServer@617] - Established session 0x35b125b84aa028a with negotiated timeout 3 for client /IP:40526 2017-06-27 21:27:38,763 [myid:] - INFO [main:QuorumPeerConfig@103] - Reading configuration from: /opt/zookeeper/current/bin/../conf/zoo.cfg 2017-06-27 21:27:38,780 [myid:] - INFO [main:QuorumPeerConfig@340] - Defaulting to majority quorums 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@78] - autopurge.snapRetainCount set to 3 2017-06-27 21:27:38,784 [myid:3] - INFO [main:DatadirCleanupManager@79] - autopurge.purgeInterval set to 1 2017-06-27 21:27:38,799 [myid:3] - INFO [PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task started. 2017-06-27 21:27:38,821 [myid:3] - INFO [main:QuorumPeerMain@127] - Starting quorum peer 2017-06-27 21:27:38,869 [myid:3] - INFO [main:NIOServerCnxnFactory@94] - binding to port /IP:2181 2017-06-27 21:27:38,
Re: java.io.EOFException
It think it was already established that a txn log file was corrupted. I'm guessing it couldn't find a combination of snapshot + txn log file that made it load the database correctly. Which version are you using? I would have been useful to check the txn log files and snapshots before getting rid of them. In the case you're interested, you can parse them using log formatter and snapshot formatter, respectively. Those are tools we make available with the distribution. -Flavio > On 29 May 2017, at 10:02, I PVP <i...@hotmail.com> wrote: > > Martin Gainty, > > Thanks for answering. > > I was able to solve the issue by running the following: > > sudo rm -rf /tmp/zookeeper > sudo rm -rf /var/lib/zookeeper > sudo rm -rf /var/log/zookeeper > sudo mkdir -p /tmp/zookeeper > sudo mkdir -p /var/lib/zookeeper/data > sudo mkdir -p /var/lib/zookeeper/data-log > sudo mkdir -p /var/log/zookeeper > sudo chown -R zookeeper:zookeeper /var/log/zookeeper/ /var/lib/zookeeper/ > /tmp/zookeeper/ > > best, > > IPVP > > > On May 28, 2017 at 9:56:43 AM, Martin Gainty > (mgai...@hotmail.com<mailto:mgai...@hotmail.com>) wrote: > > MG>below > > > From: I PVP <i...@hotmail.com> > Sent: Saturday, May 27, 2017 12:07 AM > To: UserZooKeeper > Subject: java.io.EOFException > > How to recover from the following error that started happening after a server > crash? > Zookeeper won’t start and the following message is showing repeatedly on the > log. > > 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server > environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server > environment:java.io.tmpdir=/tmp > 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server > environment:java.compiler= > 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server > environment:os.name=Linux > 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server > environment:os.arch=amd64 > 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server > environment:os.version=3.10.0-514.16.1.el7.x86_64 > 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server > environment:user.name=zookeeper > 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server > environment:user.home=/opt/zookeeper > 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server > environment:user.dir=/ > 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@829] - tickTime > set to 2000 > 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@838] - > minSessionTimeout set to -1 > 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@847] - > maxSessionTimeout set to -1 > 2017-05-27 01:02:08,080 [myid:] - INFO [main:NIOServerCnxnFactory@89] - > binding to port 0.0.0.0/0.0.0.0:2181 > 2017-05-27 01:02:08,385 [myid:] - ERROR [main:Util@239] - Last transaction > was partial. > 2017-05-27 01:02:08,400 [myid:] - ERROR [main:Util@239] - Last transaction > was partial. > 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction > was partial. > 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction > was partial. > 2017-05-27 01:02:08,404 [myid:] - ERROR [main:Util@239] - Last transaction > was partial. > 2017-05-27 01:02:08,404 [myid:] - ERROR [main:ZooKeeperServerMain@64] - > Unexpected exception, exiting abnormally > java.io.EOFException > at java.io.DataInputStream.readInt(DataInputStream.java:392) > at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > at > org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64) > at > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585) > > MG>in the logs you should see > MG>"Created new input stream" NameOfLogFile > MG>once the archive is created you should see > MG>"Created new input archive " logFile > > at > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604) > at > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570) > at > org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652) > at > org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166) > at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) > at > org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283)
Re: java.io.EOFException
Martin Gainty, Thanks for answering. I was able to solve the issue by running the following: sudo rm -rf /tmp/zookeeper sudo rm -rf /var/lib/zookeeper sudo rm -rf /var/log/zookeeper sudo mkdir -p /tmp/zookeeper sudo mkdir -p /var/lib/zookeeper/data sudo mkdir -p /var/lib/zookeeper/data-log sudo mkdir -p /var/log/zookeeper sudo chown -R zookeeper:zookeeper /var/log/zookeeper/ /var/lib/zookeeper/ /tmp/zookeeper/ best, IPVP On May 28, 2017 at 9:56:43 AM, Martin Gainty (mgai...@hotmail.com<mailto:mgai...@hotmail.com>) wrote: MG>below From: I PVP <i...@hotmail.com> Sent: Saturday, May 27, 2017 12:07 AM To: UserZooKeeper Subject: java.io.EOFException How to recover from the following error that started happening after a server crash? Zookeeper won’t start and the following message is showing repeatedly on the log. 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler= 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:os.version=3.10.0-514.16.1.el7.x86_64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.name=zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/opt/zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/ 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@829] - tickTime set to 2000 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@838] - minSessionTimeout set to -1 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@847] - maxSessionTimeout set to -1 2017-05-27 01:02:08,080 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 2017-05-27 01:02:08,385 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,400 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:ZooKeeperServerMain@64] - Unexpected exception, exiting abnormally java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585) MG>in the logs you should see MG>"Created new input stream" NameOfLogFile MG>once the archive is created you should see MG>"Created new input archive " logFile at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166) at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283) at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410) at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) any help is appreciate best IPVP
Re: java.io.EOFException
MG>below From: I PVP <i...@hotmail.com> Sent: Saturday, May 27, 2017 12:07 AM To: UserZooKeeper Subject: java.io.EOFException How to recover from the following error that started happening after a server crash? Zookeeper won’t start and the following message is showing repeatedly on the log. 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler= 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:os.version=3.10.0-514.16.1.el7.x86_64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.name=zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/opt/zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/ 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@829] - tickTime set to 2000 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@838] - minSessionTimeout set to -1 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@847] - maxSessionTimeout set to -1 2017-05-27 01:02:08,080 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 2017-05-27 01:02:08,385 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,400 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:ZooKeeperServerMain@64] - Unexpected exception, exiting abnormally java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585) MG>in the logs you should see MG>"Created new input stream" NameOfLogFile MG>once the archive is created you should see MG>"Created new input archive " logFile at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166) at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283) at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410) at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) any help is appreciate best IPVP
java.io.EOFException
How to recover from the following error that started happening after a server crash? Zookeeper won’t start and the following message is showing repeatedly on the log. 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.io.tmpdir=/tmp 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:java.compiler= 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.name=Linux 2017-05-27 01:02:08,072 [myid:] - INFO [main:Environment@100] - Server environment:os.arch=amd64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:os.version=3.10.0-514.16.1.el7.x86_64 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.name=zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.home=/opt/zookeeper 2017-05-27 01:02:08,073 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/ 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@829] - tickTime set to 2000 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@838] - minSessionTimeout set to -1 2017-05-27 01:02:08,074 [myid:] - INFO [main:ZooKeeperServer@847] - maxSessionTimeout set to -1 2017-05-27 01:02:08,080 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181 2017-05-27 01:02:08,385 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,400 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,403 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:Util@239] - Last transaction was partial. 2017-05-27 01:02:08,404 [myid:] - ERROR [main:ZooKeeperServerMain@64] - Unexpected exception, exiting abnormally java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) at org.apache.zookeeper.server.persistence.FileHeader.deserialize(FileHeader.java:64) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.inStreamCreated(FileTxnLog.java:585) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.createInputArchive(FileTxnLog.java:604) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.goToNextLog(FileTxnLog.java:570) at org.apache.zookeeper.server.persistence.FileTxnLog$FileTxnIterator.next(FileTxnLog.java:652) at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:166) at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223) at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:283) at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:410) at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:118) at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:119) at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87) at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53) at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116) at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78) any help is appreciate best IPVP
