rdhabalia opened a new pull request #2993: Handle unknown runtime exception while reading entries URL: https://github.com/apache/pulsar/pull/2993 ### Motivation Recently we have seen below runtime-exception while reading entries at broker and broker is not handling it correctly and not completing read callback so, dispatch get stuck for the subscription. ``` 22:16:58.424 [bookkeeper-ml-workers-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught java.lang.NullPointerException: null at org.apache.bookkeeper.mledger.impl.EntryImpl.create(EntryImpl.java:89) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:225) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.internalReadFromLedger(ManagedLedgerImpl.java:1509) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntries(ManagedLedgerImpl.java:1359) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.notifyEntriesAvailable(ManagedCursorImpl.java:2221) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$notifyCursors$27(ManagedLedgerImpl.java:1580) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.7.2.jar:4.7.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.22.Final.jar:4.1.22.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] ``` ``` 3:22:26.425 [bookkeeper-ml-workers-OrderedExecutor-10-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught java.lang.NullPointerException: null at org.apache.bookkeeper.mledger.impl.EntryImpl.create(EntryImpl.java:89) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:225) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.internalReadFromLedger(ManagedLedgerImpl.java:1509) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntries(ManagedLedgerImpl.java:1359) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.OpReadEntry.lambda$checkReadCompletion$76(OpReadEntry.java:136) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] 06:58:57.334 [bookkeeper-ml-workers-OrderedExecutor-19-0] ERROR org.apache.bookkeeper.common.util.SafeRunnable - Unexpected throwable caught java.lang.NullPointerException: null at org.apache.bookkeeper.mledger.impl.EntryImpl.create(EntryImpl.java:89) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:225) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.internalReadFromLedger(ManagedLedgerImpl.java:1509) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntries(ManagedLedgerImpl.java:1359) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.notifyEntriesAvailable(ManagedCursorImpl.java:2221) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$notifyCursors$27(ManagedLedgerImpl.java:1580) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[managed-ledger-original-2.2.2-yahoo.jar:2.2.2-yahoo] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.7.2.jar:4.7.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [pulsar-functions-metrics-2.2.2-yahoo.jar:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_131] ``` ### Modifications Handle NPE and complete the callback with failure. ### Result Dispatch will not be blocked if there is runtime exception while reading entries.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
