Build failed in Jenkins: bookkeeper-trunk #645
See https://builds.apache.org/job/bookkeeper-trunk/645/ -- [...truncated 513 lines...] [INFO] No sources to compile [INFO] [INFO] --- maven-surefire-plugin:2.9:test (default-test) @ bookkeeper-stats-api --- [INFO] Surefire report directory: https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-stats/target/surefire-reports --- T E S T S --- --- T E S T S --- Results : Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @ bookkeeper-stats-api --- [INFO] Building jar: https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-stats/target/bookkeeper-stats-api-4.3.0-SNAPSHOT.jar [INFO] [INFO] findbugs-maven-plugin:2.5.2:check (default-cli) @ bookkeeper-stats-api [INFO] [INFO] --- findbugs-maven-plugin:2.5.2:findbugs (findbugs) @ bookkeeper-stats-api --- [INFO] Fork Value is true [INFO] Done FindBugs Analysis [INFO] [INFO] findbugs-maven-plugin:2.5.2:check (default-cli) @ bookkeeper-stats-api [INFO] [INFO] --- findbugs-maven-plugin:2.5.2:check (default-cli) @ bookkeeper-stats-api --- [INFO] BugInstance size is 0 [INFO] Error size is 0 [INFO] No errors/warnings found [INFO] [INFO] [INFO] Building bookkeeper-server 4.3.0-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ bookkeeper-server --- [INFO] Deleting https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-server (includes = [dependency-reduced-pom.xml], excludes = []) [INFO] [INFO] --- apache-rat-plugin:0.7:check (default-cli) @ bookkeeper-server --- [INFO] Exclude: **/DataFormats.java [INFO] [INFO] --- maven-remote-resources-plugin:1.1:process (default) @ bookkeeper-server --- [INFO] [INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ bookkeeper-server --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 3 resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ bookkeeper-server --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 174 source files to https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-server/target/classes [INFO] [INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ bookkeeper-server --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @ bookkeeper-server --- [INFO] Changes detected - recompiling the module! [INFO] Compiling 84 source files to https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-server/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.9:test (default-test) @ bookkeeper-server --- [INFO] Surefire report directory: https://builds.apache.org/job/bookkeeper-trunk/ws/bookkeeper-server/target/surefire-reports --- T E S T S --- --- T E S T S --- Running org.apache.bookkeeper.client.SlowBookieTest Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.513 sec Running org.apache.bookkeeper.client.ListLedgersTest Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.295 sec Running org.apache.bookkeeper.client.BookieRecoveryTest Tests run: 72, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.694 sec Running org.apache.bookkeeper.client.TestReadTimeout Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.406 sec Running org.apache.bookkeeper.client.LedgerRecoveryTest Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.099 sec Running org.apache.bookkeeper.client.BookKeeperTest Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.904 sec Running org.apache.bookkeeper.client.RoundRobinDistributionScheduleTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.118 sec Running org.apache.bookkeeper.client.BookKeeperCloseTest Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.471 sec Running org.apache.bookkeeper.client.TestFencing Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.358 sec Running org.apache.bookkeeper.client.TestLedgerChecker Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.338 sec Running org.apache.bookkeeper.client.TestRackawareEnsemblePlacementPolicy Tests run: 7,
[jira] [Commented] (BOOKKEEPER-760) Don't close PCBC proactively if bookies disappeared from zookeeper znodes.
[
https://issues.apache.org/jira/browse/BOOKKEEPER-760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009090#comment-14009090
]
Flavio Junqueira commented on BOOKKEEPER-760:
-
If we really don't need this part of the code, I'd rather remove it altogether.
Some questions:
bq. // a. the dead bookies will be removed from topology, which will not be
used in new ensemble.
It depends on whether the session of the bookie has already expired, no?
bq. // b. the read sequence will be reordered based on znode availability, so
most of the reads will not be sent to them.
I'm not sure what reordered means here. Do you mean to say rerouted or resent?
{quote}
// c. the close here is just to disconnect the channel, which doesn't remove
the channel
//from from pcbc map. we don't really need to disconnect the channel here,
since if a bookie is
//really down, PCBC will disconnect itself based on netty callback. if we
try to disconnect
//here, it actually introduces side-effects on case d.
{quote}
Ok, this one sounds like a good reason for removing... but what are the
side-effects?
{quote}
// d. closing the client here will affect latency if the bookie is alive but
just being flaky
//on its znode registration due zookeeper session expire.
{quote}
I don't understand how it affects latency.
bq. // e. if we want to permanently remove a bookkeeper client, we should watch
on the cookies' list.
I'm a bit confused here. Do we even want to remove a bookkeeper client?
Don't close PCBC proactively if bookies disappeared from zookeeper znodes.
--
Key: BOOKKEEPER-760
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-760
Project: Bookkeeper
Issue Type: Improvement
Components: bookkeeper-client
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.3.0, 4.2.3
Attachments: BOOKKEEPER-760.diff
Don't close PCBC proactively if bookies disappeared from zookeeper znodes.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (BOOKKEEPER-760) Don't close PCBC proactively if bookies disappeared from zookeeper znodes.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14009222#comment-14009222 ] Sijie Guo commented on BOOKKEEPER-760: -- It depends on whether the session of the bookie has already expired, no? yup. if the zk session was expired, the znode of that bookie will be removed and the topology will be changed in clients, so the bookie will not be used for new ensemble. for current, a bookie will be shutdown immediately if its zookeeper session is expired. but after BOOKKEEPER-537, a bookie will re-establish the zookeeper session and re-registered itself when its previous session expired. so the change is to prevent closing channels that a bookie isn't actually down but whose session just expired, which cause unnecessary retries and ensemble changes. I don't understand how it affects latency. it would trigger unnecessary ensemble change if a bookie isn't actually down but just whose znode disappeared, so it would increase write latency. for the read latency, it would cause client re-send the read to a different bookie in same quorum, so the read latency will be increased due to retrying the second bookie in same quorum. I'm not sure what reordered means here. Do you mean to say rerouted or resent? Ah, the reorder would be in a different patch. I forgot to create the ticket for it. The reorder means we read the entries based on the availability of bookies (existence of znodes). Do we even want to remove a bookkeeper client? in theory, we might need to remove a bookie client if it is removed from the cluster permanently. but since it is rare, it is ok that we don't remove it. or, we could use a guava cache to maintain the pcbc clients, so we could evict the idle clients to avoid increasing memory. If we really don't need this part of the code, I'd rather remove it altogether. ah. I put it there with the comments so we don't add it back in future by mistake. Don't close PCBC proactively if bookies disappeared from zookeeper znodes. -- Key: BOOKKEEPER-760 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-760 Project: Bookkeeper Issue Type: Improvement Components: bookkeeper-client Reporter: Sijie Guo Assignee: Sijie Guo Fix For: 4.3.0, 4.2.3 Attachments: BOOKKEEPER-760.diff Don't close PCBC proactively if bookies disappeared from zookeeper znodes. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 17895: BOOKKEEPER-582: protobuf support for bookkeeper
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17895/#review43940 --- just a ping to @fpj and @Rakesh - Sijie Guo On April 24, 2014, 7:43 a.m., Sijie Guo wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17895/ --- (Updated April 24, 2014, 7:43 a.m.) Review request for bookkeeper and Ivan Kelly. Bugs: BOOKKEEPER-582 https://issues.apache.org/jira/browse/BOOKKEEPER-582 Repository: bookkeeper-git Description --- - introducing protobuf support for bookkeeper - for server: introduce packet processor / EnDecoder for different protocol supports - for client: change PCBC to use protobuf to send requests - misc changes for protobuf support (bookie server is able for backward compatibility) Diffs - bookkeeper-server/pom.xml ebc1198 bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexInMemPageMgr.java 56487aa bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerChecker.java 28e23d6 bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingReadOp.java fb36b90 bookkeeper-server/src/main/java/org/apache/bookkeeper/processor/RequestProcessor.java 241f369 bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieProtoEncoding.java 1154047 bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestHandler.java b922a82 bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieRequestProcessor.java 8155b22 bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperProtocol.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBase.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PacketProcessorBaseV3.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/PerChannelBookieClient.java a10f7d5 bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessor.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteEntryProcessor.java PRE-CREATION bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/WriteEntryProcessorV3.java PRE-CREATION bookkeeper-server/src/main/proto/BookkeeperProtocol.proto PRE-CREATION bookkeeper-server/src/main/resources/findbugsExclude.xml 97a6156 bookkeeper-server/src/test/java/org/apache/bookkeeper/proto/TestProtoVersions.java 5fcc445 bookkeeper-server/src/test/java/org/apache/bookkeeper/replication/AuditorPeriodicCheckTest.java 3f8496f bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieClientTest.java bc05229 bookkeeper-server/src/test/java/org/apache/bookkeeper/test/TestBackwardCompat.java 8376b46 compat-deps/bookkeeper-server-compat-4.2.0/pom.xml PRE-CREATION compat-deps/hedwig-server-compat-4.2.0/pom.xml PRE-CREATION compat-deps/pom.xml f79582d hedwig-server/pom.xml 06cf01c hedwig-server/src/test/java/org/apache/hedwig/server/TestBackwardCompat.java 8da109e Diff: https://reviews.apache.org/r/17895/diff/ Testing --- unit tests. backward tests. Thanks, Sijie Guo
