[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836672#comment-13836672 ] Ivan Kelly commented on BOOKKEEPER-710: --- Would I be correct in thinking that this fix is working in the direction of eventually having a efficient tailing mechanism for bookkeeper without polling? > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838018#comment-13838018 ] Sijie Guo commented on BOOKKEEPER-710: -- this change is fixing the correctness issue for polling. and yes, we are working on several changes to improve polling, e.g. piggyback lac when reading entries, long-polling for reading lac. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13842986#comment-13842986 ] Hadoop QA commented on BOOKKEEPER-710: -- Testing JIRA BOOKKEEPER-710 Patch [BOOKKEEPER-710.diff|https://issues.apache.org/jira/secure/attachment/12617793/BOOKKEEPER-710.diff] downloaded at Mon Dec 9 08:29:51 UTC 2013 {color:green}+1 PATCH_APPLIES{color} {color:green}+1 CLEAN{color} {color:green}+1 RAW_PATCH_ANALYSIS{color} .{color:green}+1{color} the patch does not introduce any @author tags .{color:green}+1{color} the patch does not introduce any tabs .{color:green}+1{color} the patch does not introduce any trailing spaces .{color:green}+1{color} the patch does not introduce any line longer than 120 .{color:green}+1{color} the patch does adds/modifies 2 testcase(s) {color:green}+1 RAT{color} .{color:green}+1{color} the patch does not seem to introduce new RAT warnings {color:green}+1 JAVADOC{color} .{color:green}+1{color} the patch does not seem to introduce new Javadoc warnings {color:green}+1 COMPILE{color} .{color:green}+1{color} HEAD compiles .{color:green}+1{color} patch compiles .{color:green}+1{color} the patch does not seem to introduce new javac warnings {color:green}+1 FINDBUGS{color} .{color:green}+1{color} the patch does not seem to introduce new Findbugs warnings {color:green}+1 TESTS{color} .Tests run: 886 {color:green}+1 DISTRO{color} .{color:green}+1{color} distro tarball builds with the patch {color:green}*+1 Overall result, good!, no -1s*{color} The full output of the test-patch run is available at . https://builds.apache.org/job/bookkeeper-trunk-precommit-build/550/ > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13871950#comment-13871950 ] Ivan Kelly commented on BOOKKEEPER-710: --- How do you communicate the change in LAC to the client. The client will be still be polling by calling readLastAddConfirmed, no? > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873575#comment-13873575 ] Ivan Kelly commented on BOOKKEEPER-710: --- I've been thinking about this very scenario lately, and really, I think readLastAddConfirmed is just a broken api. The correct thing to do here would be to do an openNoRecovery() in the loop. This will only involve metadata reads from ZK/metastore, and these should be very fast, given they're going to memory. Have you stats on how many watches zookeeper can sustain? > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873592#comment-13873592 ] Sijie Guo commented on BOOKKEEPER-710: -- {quote} The correct thing to do here would be to do an openNoRecovery() in the loop. This will only involve metadata reads from ZK/metastore, and these should be very fast, given they're going to memory. {quote} you could implement this polling mechanism in ledger manager (for metastore that doesn't support watcher/notification) to satisfy the listener interface in the patch. that's ok. but for metastore like zookeeper, they already provide watcher, which we could leverage to reduce polling traffic to zookeeper. since ensemble change only happens during bookie node failures, e.g rolling restart, which is a very rare in most of case. it is not good to introduce too much traffic to zookeeper. {quote} Have you stats on how many watches zookeeper can sustain? {quote} no. we don't have this number. I assumed that it depends on the memory of zookeeper. [~fpj] might have idea on how many watches zookeeper can sustain. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874994#comment-13874994 ] Hudson commented on BOOKKEEPER-710: --- SUCCESS: Integrated in bookkeeper-trunk #512 (See [https://builds.apache.org/job/bookkeeper-trunk/512/]) BOOKKEEPER-710: OpenLedgerNoRecovery should watch ensemble change. (sijie via ivank) (ivank: rev 1559192) * /zookeeper/bookkeeper/trunk/CHANGES.txt * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerOpenOp.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/ReadOnlyLedgerHandle.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/LedgerManager.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/MSLedgerManagerFactory.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookkeeperInternalCallbacks.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/replication/BookieLedgerIndexer.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java * /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestWatchEnsembleChange.java > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932022#comment-13932022 ] Sijie Guo commented on BOOKKEEPER-710: -- [~ikelly] I marked this for 4.2.3 when I created the ticket. this is a bug fix not a new feature. we should get it in 4.2.3. but I have no idea why it is on reopened state. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932045#comment-13932045 ] Ivan Kelly commented on BOOKKEEPER-710: --- I reopened, to mark it as non-resolved for 4.2.3. I think everything else in 4.2.3 has been put in the 4.2 branch > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932053#comment-13932053 ] Rakesh R commented on BOOKKEEPER-710: - [~hustlmsp] since its not merged into 4.2.3 branch, so its better to be in open state until we finish the changes and this will remind us. I also feel, it can go into 4.2.3 as well. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932067#comment-13932067 ] Sijie Guo commented on BOOKKEEPER-710: -- [~rakeshr] I know that. I just say I don't who reopen it. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957652#comment-13957652 ] Hadoop QA commented on BOOKKEEPER-710: -- Testing JIRA BOOKKEEPER-710 Patch [0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch|https://issues.apache.org/jira/secure/attachment/12638255/0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch] downloaded at Wed Apr 2 13:41:14 UTC 2014 {color:red}-1{color} Patch failed to apply to head of branch > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: > 0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch, > BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13957660#comment-13957660 ] Ivan Kelly commented on BOOKKEEPER-710: --- qa fails since the patch is on branch-4.2 > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: > 0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch, > BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13958909#comment-13958909 ] Flavio Junqueira commented on BOOKKEEPER-710: - bq. I think readLastAddConfirmed is just a broken api I've just noticed some of the comments here, in particular this one. Is there a jira for this or this just a rant? bq. might have idea on how many watches zookeeper can sustain. It is mainly memory, but triggering many watches at a time also causes a burst of notifications. I can't think of anything else right now. I was also wondering if there is a jira for these todos: {noformat} // TODO: should provide ledger metadata listener in metadata store. {noformat} Also, do we really have to wait 5 seconds in the test to make sure that the call doesn't get stuck? {noformat} TimeUnit.SECONDS.sleep(5); {noformat} > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: > 0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch, > BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960130#comment-13960130 ] Ivan Kelly commented on BOOKKEEPER-710: --- {quote} bq. I think readLastAddConfirmed is just a broken api I've just noticed some of the comments here, in particular this one. Is there a jira for this or this just a rant? {quote} Currently no jira, but we're looking at ways to remove the need to poll. Will create a jira when's it has solidified a bit. > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: > 0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch, > BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-710) OpenLedgerNoRecovery should watch ensemble change.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966330#comment-13966330 ] Flavio Junqueira commented on BOOKKEEPER-710: - About the question on watches, I knew I had looked into it, but I couldn't remember where the discussion. In Chapter 4 of the ZK book, there is a discussion about the scalability of watches. Each new watch consumes around 300 bytes on the server, so again watches induce memory utilization on the server and possibly bursts of traffic depending on the number of notifications triggered. One way to spread watches across servers is to use observers > OpenLedgerNoRecovery should watch ensemble change. > -- > > Key: BOOKKEEPER-710 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-710 > Project: Bookkeeper > Issue Type: Bug > Components: bookkeeper-client >Reporter: Sijie Guo >Assignee: Sijie Guo >Priority: Blocker > Fix For: 4.3.0, 4.2.3 > > Attachments: > 0001-BOOKKEEPER-710-OpenLedgerNoRecovery-should-watch-ens.patch, > BOOKKEEPER-710.diff > > > LedgerHandle opened by openLedgerNoRecovery should watch ensemble change. > otherwise, readLastConfirmed & readEntries will stuck if there are ensemble > changes in the ledger. -- This message was sent by Atlassian JIRA (v6.2#6252)