zhaohaidao commented on issue #18163:
URL: https://github.com/apache/pulsar/issues/18163#issuecomment-1295030760
The error log context is as followed. Not sure if it can help.
@zymap
> 11338 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.RackawareEn
11338 semblePlacementPolicyImpl - Failed to choose a bookie: excluded
[<Bookie:10.225.50.43:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.224.145.38:3181>], fallback to choose bookie
11338 randomly from the cluster.
11339 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.RackawareEn
11339 semblePlacementPolicyImpl - Failed to find 1 bookies :
excludeBookies [<Bookie:10.225.50.43:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.224.145.38:3181>], allBookies [<Bookie:
11339 10.225.105.4:3181>, <Bookie:10.225.50.43:3181>,
<Bookie:10.224.145.38:3181>].
11340 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.RackawareEn
11340 semblePlacementPolicyImpl - Failed to find 1 bookies :
excludeBookies [<Bookie:10.225.50.43:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.224.145.38:3181>], allBookies [<Bookie:
11340 10.224.145.38:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.225.50.43:3181>].
11341 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.RackawareEn
11341 semblePlacementPolicyImpl - Failed to choose a bookie: excluded
[<Bookie:10.225.50.43:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.224.145.38:3181>], fallback to choose bookie
11341 randomly from the cluster.
11342 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.RackawareEn
11342 semblePlacementPolicyImpl - Failed to find 1 bookies :
excludeBookies [<Bookie:10.225.50.43:3181>, <Bookie:10.225.105.4:3181>,
<Bookie:10.224.145.38:3181>], allBookies [<Bookie:
11342 10.225.105.4:3181>, <Bookie:10.224.145.38:3181>,
<Bookie:10.225.50.43:3181>].
11343 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0]
ERROR org.apache.bookkeeper.client.ReadOnlyLed
11343 gerHandle - Could not get additional bookie to remake ensemble,
closing ledger: 3116
11344 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0]
ERROR org.apache.bookkeeper.client.PendingAddO
11344 p - Write of ledger entry to quorum failed: L3116 E1032
11345 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,055+0800 [BookKeeperClientWorker-OrderedExecutor-22-0]
ERROR org.apache.bookkeeper.client.LedgerRecov
11345 eryOp - Failure NotEnoughBookiesException: Not enough non-faulty
bookies available while writing entry: 1033 while recovering ledger: 3116
11346 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-13-0]
ERROR org.apache.bookkeeper.mledger.impl.Manag
11346 edLedgerImpl -
[benchmark/ns-test/persistent/test-test-reusing-topic500-0000-partition-155]
Failed to open ledger 3116: Error while recovering ledger
11347 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-13-0]
ERROR org.apache.bookkeeper.mledger.impl.Manag
11347 edLedgerFactoryImpl -
[benchmark/ns-test/persistent/test-test-reusing-topic500-0000-partition-155]
Failed to initialize managed ledger: Error while recovering ledger
11348 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-13-0] INFO
org.apache.bookkeeper.mledger.impl.Manag
11348 edLedgerImpl -
[benchmark/ns-test/persistent/test-test-reusing-topic500-0000-partition-155]
Closing managed ledger
11349 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-13-0] WARN
org.apache.pulsar.broker.service.BrokerS
11349 ervice - Failed to create topic
persistent://benchmark/ns-test/test-test-reusing-topic500-0000-partition-155
11350 Oct 27 18:02:42 test_host pulsar[166925]:
org.apache.bookkeeper.mledger.ManagedLedgerException: Error while recovering
ledger
11351 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-22-0]
ERROR org.apache.bookkeeper.common.util.SafeRu
11351 nnable - Unexpected throwable caught
11352 Oct 27 18:02:42 test_host pulsar[166925]:
java.lang.NullPointerException: Cannot read field "distributionSchedule"
because "this.lh" is null
11353 Oct 27 18:02:42 test_host pulsar[166925]: at
org.apache.bookkeeper.client.PendingAddOp.safeRun(PendingAddOp.java:272)
~[org.apache.bookkeeper-bookkeeper-server-4.1 11353
6.1-SNAPSHOT.jar:4.16.1-SNAPSHOT]
11354 Oct 27 18:02:42 test_host pulsar[166925]: at
org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
~[org.apache.bookkeeper-bookkeeper-common-4.1 11354
6.1-SNAPSHOT.jar:4.16.1-SNAPSHOT]
11355 Oct 27 18:02:42 test_host pulsar[166925]: at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
~[?:?]
11356 Oct 27 18:02:42 test_host pulsar[166925]: at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
~[?:?]
11357 Oct 27 18:02:42 test_host pulsar[166925]: at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
~[io.netty-netty-common-4.1.77.F 11357 inal.jar:4.1.77.Final]
11358 Oct 27 18:02:42 test_host pulsar[166925]: at
java.lang.Thread.run(Thread.java:833) ~[?:?]
11359 Oct 27 18:02:42 test_host pulsar[166925]:
2022-10-27T18:02:42,056+0800 [BookKeeperClientWorker-OrderedExecutor-22-0] WARN
org.apache.bookkeeper.client.LedgerRecov 11359 eryOp - Successfully read
entry 1033 for ledger 3116, but readDone is already false
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
