Siddharth Sunil Boobna created BOOKKEEPER-893:
-------------------------------------------------
Summary: bookie exited with status 0 on journal I/O exception
Key: BOOKKEEPER-893
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-893
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Affects Versions: 4.3.2
Reporter: Siddharth Sunil Boobna
Assignee: Siddharth Sunil Boobna
Priority: Minor
{noformat}
06:13:32.041 [BookieJournal-3181] ERROR org.apache.bookkeeper.bookie.Journal -
I/O exception in Journal thread!
java.nio.channels.ClosedChannelException: null
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
~[na:1.8.0_60]
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:253)
~[na:1.8.0_60]
at org.apache.bookkeeper.bookie.Journal.run(Journal.java:964)
~[bookkeeper-server-4.3.1.33.jar:4.3.1]
06:13:32.041 [BookieJournal-3181] INFO org.apache.bookkeeper.bookie.Journal -
Journal exited loop!
06:13:32.041 [Bookie-3181] ERROR org.apache.bookkeeper.bookie.Bookie - Journal
manager quits unexpectedly.
06:13:32.042 [Bookie-3181] INFO org.apache.bookkeeper.bookie.Bookie -
Triggering shutdown of Bookie-3181 with exitCode 5
06:13:32.042 [BookieShutdownTrigger] INFO org.apache.bookkeeper.bookie.Bookie
- Shutting down Bookie-3181 with exitCode 5
06:13:32.044 [main] INFO o.a.bookkeeper.proto.BookieServer - Stop stats
provider
06:13:32.045 [bookie-reg-0] INFO org.apache.zookeeper.ZooKeeper -
Session: 0x14ff63dc02cbd02 closed
06:13:32.045 [bookie-reg-0-EventThread] INFO org.apache.zookeeper.ClientCnxn
- EventThread shut down
06:13:32.045 [BookieShutdownTrigger] INFO org.apache.bookkeeper.bookie.Journal
- Shutting down Journal
06:13:32.046 [Thread-2] INFO o.a.bookkeeper.proto.BookieServer - Shutting
down BookieServer
06:13:32.046 [Thread-2] INFO o.a.b.proto.BookieNettyServer - Shutting
down BookieNettyServer
06:13:32.046 [ForceWriteThread] ERROR org.apache.bookkeeper.bookie.Journal -
ForceWrite thread interrupted
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
~[na:1.8.0_60]
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
~[na:1.8.0_60]
at
org.apache.bookkeeper.util.UnboundArrayBlockingQueue.take(UnboundArrayBlockingQueue.java:312)
~[bookkeeper-server-4.3.1.33.jar:4.3.1]
at
org.apache.bookkeeper.bookie.Journal$ForceWriteThread.run(Journal.java:448)
~[bookkeeper-server-4.3.1.33.jar:4.3.1]
...
06:13:32.184 [bookie-io-0] WARN i.n.channel.DefaultChannelPipeline - An
exceptionCaught() event was fired, and it reached at the tail of the pipeline.
It usually means the last handler in the pipeline did not handle the exception.
java.nio.channels.ClosedChannelException: null
{noformat}
When the bookie shuts down due to some exception, it exits with exitCode 0.
We try to get the exitCode when calling System.exit() here:
https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java#L400
But the exitCode is set by the shutdown hook which is triggered by
System.exit(). Thus, we will always exit with code 0.
Solution:
We explicitly call bookie shutdown to set the exit code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
