[jira] [Commented] (ZOOKEEPER-1412) java client watches inconsistently triggered on reconnect
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228483#comment-13228483 ] Benjamin Reed commented on ZOOKEEPER-1412: -- i agree with botond that we should also fix the java client to update lastzxid on every message. it might be worth doing in a separate jira. java client watches inconsistently triggered on reconnect - Key: ZOOKEEPER-1412 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1412 Project: ZooKeeper Issue Type: Bug Components: server Affects Versions: 3.3.3, 3.3.4, 3.4.0, 3.4.1, 3.4.2, 3.4.3 Reporter: Botond Hejj Assignee: Patrick Hunt Priority: Blocker Fix For: 3.3.5, 3.4.4, 3.5.0 Attachments: ZOOKEEPER-1412_br33.patch, ZOOKEEPER-1412_br33.patch, ZOOKEEPER-1412_br34.patch, ZOOKEEPER-1412_br34.patch, ZOOKEEPER-1412_trunk.patch, ZOOKEEPER-1412_trunk.patch I've observed an inconsistent behavior in java client watches. The inconsistency relates to the behavior after the client reconnects to the zookeeper ensemble. After the client reconnects to the ensemble only those watches should trigger which should have been triggered also if the connections was not lost. This means if I watch for changes in node /foo and there is no change there than my watch should not be triggered on reconnecting to the ensemble. This is not always the case in the java client. I've debugged the issues and I could locate the case when the watch is always triggered on reconnect. This is consistently happening if I connect to a follower in the ensemble and I don't do any operation which goes through the leader. Looking at the code I see that the client stores the lastzxid and sends that with its request. This is 0 on startup and will be updated everytime from the server replies. This lastzxid is also sent to the server after reconnect together with watches. The server decides which watch to trigger based on this lastzxid probably because that should mean the last known state of the client. If this lastzxid is 0 than all the watches are triggered. I've checked why is this lastzxid 0. I thought it shouldn't be since there was already a request to the server to set the watch and in the reply the server could have sent back the zxid but it turns out that it sends just 0. Looking at the server code I see that for requests which doesn't go through the leader the follower server just sends back the same zxid that the client sent. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1344) ZooKeeper client multi-update command is not considering the Chroot request
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228485#comment-13228485 ] Hadoop QA commented on ZOOKEEPER-1344: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12518195/ZOOKEEPER-1344_trunk.patch against trunk revision 1297740. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//console This message is automatically generated. ZooKeeper client multi-update command is not considering the Chroot request --- Key: ZOOKEEPER-1344 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1344 Project: ZooKeeper Issue Type: Bug Components: java client Affects Versions: 3.4.0 Reporter: Rakesh R Assignee: Rakesh R Priority: Critical Fix For: 3.5.0 Attachments: ZOOKEEPER-1344-onlytestcase.patch, ZOOKEEPER-1344.1.patch, ZOOKEEPER-1344.patch, ZOOKEEPER-1344_branch_3.4.patch, ZOOKEEPER-1344_trunk.patch For example: I have created a ZooKeeper client with subtree as 10.18.52.144:2179/apps/X. Now just generated OP command for the creation of zNode /myId. When the client creates the path /myid, the ZooKeeper server is actually be creating the path as /myid instead of creating as /apps/X/myid Expected output: zNode has to be created as /apps/X/myid -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Success: ZOOKEEPER-1344 PreCommit Build #992
Jira: https://issues.apache.org/jira/browse/ZOOKEEPER-1344 Build: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 168250 lines...] [exec] BUILD SUCCESSFUL [exec] Total time: 0 seconds [exec] [exec] [exec] [exec] [exec] +1 overall. Here are the results of testing the latest attachment [exec] http://issues.apache.org/jira/secure/attachment/12518195/ZOOKEEPER-1344_trunk.patch [exec] against trunk revision 1297740. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 core tests. The patch passed core unit tests. [exec] [exec] +1 contrib tests. The patch passed contrib unit tests. [exec] [exec] Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//testReport/ [exec] Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html [exec] Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/992//console [exec] [exec] This message is automatically generated. [exec] [exec] [exec] == [exec] == [exec] Adding comment to Jira. [exec] == [exec] == [exec] [exec] [exec] Comment added. [exec] 8D4ghhhRDE logged out [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == [exec] [exec] BUILD SUCCESSFUL Total time: 24 minutes 43 seconds Archiving artifacts Recording test results Description set: ZOOKEEPER-1344 Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Created] (ZOOKEEPER-1413) Use on-disk transaction log for learner sync up
Use on-disk transaction log for learner sync up --- Key: ZOOKEEPER-1413 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1413 Project: ZooKeeper Issue Type: Improvement Components: server Affects Versions: 3.4.3 Reporter: Thawan Kooburat Priority: Minor Motivation: The learner syncs up with leader by retrieving committed log from the leader. Currently, the leader only keeps 500 entries of recently committed log in memory. If the learner falls behind more than 500 updates, the leader will send the entire snapshot to the learner. With the size of the snapshot for some of our Zookeeper deployments (~10G), it is prohibitively expensive to send the entire snapshot over network. Additionally, our Zookeeper may serve more than 4K updates per seconds. As a result, a network hiccups for less than a second will cause the learner to use snapshot transfer. Design: Instead of looking only at committed log in memory, the leader will also look at transaction log on disk. The amount of transaction log kept on disk is configurable and the current default is 100k. This will allow Zookeeper to tolerate longer temporal network failure before initiating the snapshot transfer. Implementation: We plan to add interface to the persistence layer will can be use to retrieve proposals from on-disk transaction log. These proposals can then be used to send to the learner using existing protocol. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ZOOKEEPER-1355) Add zk.updateServerList(newServerList)
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marshall McMullen updated ZOOKEEPER-1355: - Attachment: ZOOKEEPER-1355-ver10-4.patch Uploading a new version that fixes the C code to properly force a disconnect if the list of servers is modified and the host we were connected to is not in the list anymore. Add zk.updateServerList(newServerList) --- Key: ZOOKEEPER-1355 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1355 Project: ZooKeeper Issue Type: New Feature Components: java client Reporter: Alexander Shraer Assignee: Alexander Shraer Fix For: 3.5.0 Attachments: ZOOKEEPER-1355-ver10-1.patch, ZOOKEEPER-1355-ver10-2.patch, ZOOKEEPER-1355-ver10-3.patch, ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10-4.patch, ZOOKEEPER-1355-ver10.patch, ZOOKEEPER-1355-ver2.patch, ZOOKEEPER-1355-ver4.patch, ZOOKEEPER-1355-ver5.patch, ZOOKEEPER-1355-ver6.patch, ZOOKEEPER-1355-ver7.patch, ZOOKEEPER-1355-ver8.patch, ZOOKEEPER-1355-ver9-1.patch, ZOOKEEPER-1355-ver9.patch, ZOOKEEPER=1355-ver3.patch, ZOOOKEEPER-1355-test.patch, ZOOOKEEPER-1355-ver1.patch, ZOOOKEEPER-1355.patch, loadbalancing-more-details.pdf, loadbalancing.pdf When the set of servers changes, we would like to update the server list stored by clients without restarting the clients. Moreover, assuming that the number of clients per server is the same (in expectation) in the old configuration (as guaranteed by the current list shuffling for example), we would like to re-balance client connections across the new set of servers in a way that a) the number of clients per server is the same for all servers (in expectation) and b) there is no excessive/unnecessary client migration. It is simple to achieve (a) without (b) - just re-shuffle the new list of servers at every client. But this would create unnecessary migration, which we'd like to avoid. We propose a simple probabilistic migration scheme that achieves (a) and (b) - each client locally decides whether and where to migrate when the list of servers changes. The attached document describes the scheme and shows an evaluation of it in Zookeeper. We also implemented re-balancing through a consistent-hashing scheme and show a comparison. We derived the probabilistic migration rules from a simple formula that we can also provide, if someone's interested in the proof. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-163) Prevent incorrect NoSuchLedgerException for readLastConfirmed.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228247#comment-13228247 ] Sijie Guo commented on BOOKKEEPER-163: -- committed as r1299984. Great work, Thanks Ivan. Prevent incorrect NoSuchLedgerException for readLastConfirmed. -- Key: BOOKKEEPER-163 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-163 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-client, bookkeeper-server Affects Versions: 4.0.0 Reporter: Sijie Guo Assignee: Ivan Kelly Fix For: 4.1.0 Attachments: BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff, BOOKKEEPER-163.diff bookkeeper client treats NoSuchLedgerException as valid response when reading last confirmed. If NoSuchLedgerException is caused due to an empty directory in following cases, it is an incorrect response. 1) A disk is replaced or ledger index is removed by a sloppy admin. 2) A disk is not mounted when a bookie machine is restarted. We need a mechanism to prevent such incorrect responses. Ivan suggested to generate a instance key for each bookie and write it into the ledger directories. If a directory doesn't have the key, and other directories do, then it shouldn't start. This would also resolve the issue that someone starting a new bookie with the same IP as a bookie which has previously died. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (BOOKKEEPER-184) CompactionTest failing on Jenkins
CompactionTest failing on Jenkins - Key: BOOKKEEPER-184 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Reporter: Sijie Guo Assignee: Sijie Guo Fix For: 4.1.0 the compaction test hanging on Jenkins. Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this. 1. Only create 1 bookie 2. Not inherit from BaseTestCase 3. Reduce the number of entries -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-25) Implementation of the JMS specification and API
[ https://issues.apache.org/jira/browse/BOOKKEEPER-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated BOOKKEEPER-25: --- Fix Version/s: (was: 4.1.0) 4.2.0 Implementation of the JMS specification and API --- Key: BOOKKEEPER-25 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-25 Project: Bookkeeper Issue Type: Task Reporter: Matthieu Morel Fix For: 4.2.0 Attachments: JMS implementation.pdf, JMS implementation.pdf, hedwigjms-milestone1preview.pdf Here is a proposal for implementing JMS on top of Hedwig. Overview: - After comparing the JMS spec and Hedwig's current implementation, it appears that Hedwig provides most concepts of the JMS spec. The work will therefore mostly be implementing wrappers around Hedwig's primitives. There is no 1 to 1 mapping most of the time though, and there are some subtleties in the JMS spec that we must be cautious about. There are also things that don't exist in Hedwig and that we'll have to add. For instance, administered objects, the different kinds of standard messages, or a parser for the SQL-like selector queries. These are marked with a triangular flag on the attached mindmap. The JMS spec also specifies support for distributed transactions (XA) but this part is optional. I would suggest to leave that for later, if there is any request for it. Design: --- I suggest to separate all JMS-related code in a separate project (as hedwig-client and hedwig-protocol). We'll therefore make sure that Hedwig-server is JMS agnostic, and it will later be possible to use the same approach for implementing AMQP or JMS2, without mixing all concepts. Development: --- An iterative approach is well suited. We could start with fundamentals, and progressively add some meat: 1. messages, connections and sessions 2. basic messaging with selectors 3. all cases of messaging models 4. validation tests 5. performance tests (I checked and unfortunately, there does not seem to be any free standard for benchmarking jms implementation...) Each iteration would involve adding regression tests and doing a review. About the time this would take: My rough estimation (not being very familiar with Hedwig's codebase): in the order 1 or 2 weeks for each of these iterations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-56) Race condition of message handler in connection recovery in Hedwig client
[ https://issues.apache.org/jira/browse/BOOKKEEPER-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228308#comment-13228308 ] Flavio Junqueira commented on BOOKKEEPER-56: Should we move this one to 4.2.0? Race condition of message handler in connection recovery in Hedwig client - Key: BOOKKEEPER-56 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-56 Project: Bookkeeper Issue Type: Bug Components: hedwig-client Affects Versions: 4.0.0 Reporter: Gavin Li Assignee: Gavin Li Fix For: 4.1.0 Attachments: patch_56 There's a race condition in the connection recovery logic in Hedwig client. The message handler user set might be overwritten incorrectly. When handling channelDisconnected event, we try to reconnect to Hedwig server. After the connection is created and subscribed, we'll call StartDelivery() to recover the message handler to the original one of the disconnected connection. But if during this process, user calls StartDelivery() to set a new message handler, it will get overwritten to the original one. The process can be demonstrated as below: || main thread || netty worker thread || | StartDelivery(messageHandlerA) | | | (connection Broken here, and recovered later...) | | | ResponseHandler::channelDisconnected() (connection disconnected event received) | | | new SubscribeReconnectCallback(subHandler.getMessageHandler()) (store messageHandlerA in SubscribeReconnectCallback to recover later) | | | client.doConnect() (try reconnect) | | | doSubUnsub() (resubscribe) | | | SubscriberResponseHandler::handleSubscribeResponse() (subscription succeeds) | | StartDelivery(messageHandlderB) | | | | SubscribeReconnectCallback::operationFinished() | | | StartDelvery(messageHandlerA) (messageHandler get overwritten) | I can stably reproduce this by simulating this race condition by put some sleep in ResponseHandler. I think essentially speaking we should not store messageHandler in ResponseHandler, since the message handler is supposed to be bound to connection. Instead, no matter which connection is in use, we should use the same messageHandler, the one user set last time. So I think we should change to store messageHandler in the HedwigSubscriber, in this way we don't need to recover the handler in connection recovery and thus won't face this race condition. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-168) Message bounding on subscriptions
[ https://issues.apache.org/jira/browse/BOOKKEEPER-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated BOOKKEEPER-168: -- Attachment: BOOKKEEPER-168.diff Added new subscribe method, which takes SubscriptionOptions. This can be extended in the future to take more options if this is ever needed. Message bounding on subscriptions - Key: BOOKKEEPER-168 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-168 Project: Bookkeeper Issue Type: New Feature Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 4.1.0 Attachments: BOOKKEEPER-168.diff, BOOKKEEPER-168.diff, BOOKKEEPER-168.diff In hedwig, messages for a subscription will queue up forever if the subscriber is offline. In some usecases, this is undesirable, as it will eventually mean resource exhaustion. In this JIRA we propose an optional change to the subscription contract, which allows the user to set a bound on the number of messages which will be queued for its subscription while it is offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-168) Message bounding on subscriptions
[ https://issues.apache.org/jira/browse/BOOKKEEPER-168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228320#comment-13228320 ] jirapos...@reviews.apache.org commented on BOOKKEEPER-168: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3824/ --- (Updated 2012-03-13 11:16:09.577984) Review request for bookkeeper. Summary --- In hedwig, messages for a subscription will queue up forever if the subscriber is offline. In some usecases, this is undesirable, as it will eventually mean resource exhaustion. In this JIRA we propose an optional change to the subscription contract, which allows the user to set a bound on the number of messages which will be queued for its subscription while it is offline. This addresses bug BOOKKEEPER-168. https://issues.apache.org/jira/browse/BOOKKEEPER-168 Diffs (updated) - hedwig-client/src/main/cpp/inc/hedwig/client.h f37ef98 hedwig-client/src/main/cpp/inc/hedwig/subscribe.h 775a32c hedwig-client/src/main/cpp/lib/client.cpp 6d70ad9 hedwig-client/src/main/cpp/lib/data.h b4e2c15 hedwig-client/src/main/cpp/lib/data.cpp a223120 hedwig-client/src/main/cpp/lib/subscriberimpl.h 1cad984 hedwig-client/src/main/cpp/lib/subscriberimpl.cpp 32075c4 hedwig-client/src/main/cpp/log4cpp.conf 5651e25 hedwig-client/src/main/cpp/scripts/log4cxx.conf fc1084c hedwig-client/src/main/cpp/scripts/tester.sh 5df613a hedwig-client/src/main/cpp/test/Makefile.am 62902e6 hedwig-client/src/main/cpp/test/main.cpp 3290af3 hedwig-client/src/main/cpp/test/messageboundtest.cpp PRE-CREATION hedwig-client/src/main/java/org/apache/hedwig/client/api/Subscriber.java 6910e7d hedwig-client/src/main/java/org/apache/hedwig/client/conf/ClientConfiguration.java 66e049f hedwig-client/src/main/java/org/apache/hedwig/client/data/PubSubData.java 1b33c2f hedwig-client/src/main/java/org/apache/hedwig/client/netty/HedwigSubscriber.java 5fb1d88 hedwig-protocol/src/main/java/org/apache/hedwig/protocol/PubSubProtocol.java f46868b hedwig-protocol/src/main/protobuf/PubSubProtocol.proto e44d981 hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java 2c9af4d hedwig-server/src/main/java/org/apache/hedwig/server/persistence/BookkeeperPersistenceManager.java f1fd965 hedwig-server/src/main/java/org/apache/hedwig/server/persistence/LocalDBPersistenceManager.java 02ec607 hedwig-server/src/main/java/org/apache/hedwig/server/persistence/PersistenceManager.java 5c38ad9 hedwig-server/src/main/java/org/apache/hedwig/server/persistence/ReadAheadCache.java 0261521 hedwig-server/src/main/java/org/apache/hedwig/server/regions/HedwigHubSubscriber.java 40aa338 hedwig-server/src/main/java/org/apache/hedwig/server/subscriptions/AbstractSubscriptionManager.java 714a631 hedwig-server/src/test/java/org/apache/hedwig/server/persistence/MessageBoundedPersistenceTest.java PRE-CREATION hedwig-server/src/test/java/org/apache/hedwig/server/persistence/StubPersistenceManager.java 84b866d Diff: https://reviews.apache.org/r/3824/diff Testing --- Thanks, Ivan Message bounding on subscriptions - Key: BOOKKEEPER-168 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-168 Project: Bookkeeper Issue Type: New Feature Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 4.1.0 Attachments: BOOKKEEPER-168.diff, BOOKKEEPER-168.diff, BOOKKEEPER-168.diff In hedwig, messages for a subscription will queue up forever if the subscriber is offline. In some usecases, this is undesirable, as it will eventually mean resource exhaustion. In this JIRA we propose an optional change to the subscription contract, which allows the user to set a bound on the number of messages which will be queued for its subscription while it is offline. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-143) Add SSL support for hedwig cpp client
[ https://issues.apache.org/jira/browse/BOOKKEEPER-143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated BOOKKEEPER-143: Fix Version/s: (was: 4.1.0) 4.2.0 Moving it to 4.2.0. Add SSL support for hedwig cpp client - Key: BOOKKEEPER-143 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-143 Project: Bookkeeper Issue Type: Improvement Components: bookkeeper-client Affects Versions: 4.0.0 Reporter: Sijie Guo Fix For: 4.2.0 Currently Hedwig CPP client doesn't support SSL. Example for boost::asio using SSL: http://www.boost.org/doc/libs/1_46_1/doc/html/boost_asio/example/ssl/client.cpp -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on Jenkins
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228324#comment-13228324 ] Ivan Kelly commented on BOOKKEEPER-184: --- Did you figure out why the test cases were not failing? Before this change, the disabling of compaction in #testDisableCompaction had no affect. However, the test still passed happily, which indicates that the test wasn't correct. CompactionTest failing on Jenkins - Key: BOOKKEEPER-184 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-server Reporter: Sijie Guo Assignee: Sijie Guo Fix For: 4.1.0 Attachments: BK-184.diff the compaction test hanging on Jenkins. Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this. 1. Only create 1 bookie 2. Not inherit from BaseTestCase 3. Reduce the number of entries -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Review Request: BOOKKEEPER-96: extends zookeeper JMX to monitor and manage hedwig server
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4307/ --- Review request for bookkeeper. Summary --- we can extends / reuses zookeeper JMX utils to monitor and manage hedwig server This addresses bug BOOKKEEPER-96. https://issues.apache.org/jira/browse/BOOKKEEPER-96 Diffs - hedwig-server/src/main/java/org/apache/hedwig/server/jmx/HedwigJMXService.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/jmx/HedwigMBeanInfo.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/jmx/HedwigMBeanRegistry.java PRE-CREATION Diff: https://reviews.apache.org/r/4307/diff Testing --- Thanks, Sijie
Review Request: BOOKKEEPER-97: collect pub/sub/consume statistics on hub server
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4308/ --- Review request for bookkeeper. Summary --- collecting pub/sub/consume statistics on hub server and expose them thru JMX. the patch is based on BOOKKEEPER-96, includes BOOKKEEPER-99, exposing statistics thru JMX. This addresses bug BOOKKEEPER-97. https://issues.apache.org/jira/browse/BOOKKEEPER-97 Diffs - hedwig-server/src/main/java/org/apache/hedwig/server/delivery/FIFODeliveryManager.java 2c9af4d hedwig-server/src/main/java/org/apache/hedwig/server/handlers/BaseHandler.java 2573df0 hedwig-server/src/main/java/org/apache/hedwig/server/handlers/ConsumeHandler.java 1712027 hedwig-server/src/main/java/org/apache/hedwig/server/handlers/NettyHandlerBean.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/handlers/NettyHandlerMXBean.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/handlers/PublishHandler.java 7a785c4 hedwig-server/src/main/java/org/apache/hedwig/server/handlers/SubscribeHandler.java f3b108a hedwig-server/src/main/java/org/apache/hedwig/server/handlers/UnsubscribeHandler.java 09e8b92 hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServer.java 27e3104 hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServerBean.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/netty/PubSubServerMXBean.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/netty/ServerStats.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/netty/UmbrellaHandler.java 2f896d2 hedwig-server/src/main/java/org/apache/hedwig/server/persistence/ReadAheadCache.java 0261521 hedwig-server/src/main/java/org/apache/hedwig/server/persistence/ReadAheadCacheBean.java PRE-CREATION hedwig-server/src/main/java/org/apache/hedwig/server/persistence/ReadAheadCacheMXBean.java PRE-CREATION Diff: https://reviews.apache.org/r/4308/diff Testing --- Thanks, Sijie
[jira] [Updated] (BOOKKEEPER-55) SubscribeReconnectRetryTask might retry subscription endlessly when another subscription is already successfully created previously
[ https://issues.apache.org/jira/browse/BOOKKEEPER-55?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Flavio Junqueira updated BOOKKEEPER-55: --- Fix Version/s: (was: 4.1.0) 4.2.0 Moving it to 4.2.0. SubscribeReconnectRetryTask might retry subscription endlessly when another subscription is already successfully created previously --- Key: BOOKKEEPER-55 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-55 Project: Bookkeeper Issue Type: Bug Components: bookkeeper-client Affects Versions: 4.0.0 Reporter: Gavin Li Assignee: Gavin Li Fix For: 4.2.0 Attachments: patch For channelDisconnected envent, we try to automatically recover the connection and subscription. But when users call HedwigSubscriber.subscribe() at the same time, it might succeed before the auto recovery. Then the auto recovery can never succeed as the server will report topic busy failure. Then the SubscribeReconnectRetryTask will retry again and again endlessly. We found this in our auto test. Fix is easy, we just need to firstly check if the channel for this topic and subscribe id is null, if not it means some subscription is already created before, we don't need to bother recover. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Question about jira
I have created a new release version (4.2.0) on jira and marked a bunch of issues for this version. When I go to issues, 4.2.0 does not show up and if I write a filter for it, it says that the there is no issue with fix version 4.2.0. Does anyone know if this is a bug or if I need to change something else? -Flavio
[jira] [Created] (BOOKKEEPER-185) Remove bookkeeper-server dependency on hadoop-common
Remove bookkeeper-server dependency on hadoop-common Key: BOOKKEEPER-185 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-185 Project: Bookkeeper Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 4.1.0 bookkeeper-server depends on hadoop-common simply for the use of 1 class, HardLink. This clutters the pom.xml as a lot of stuff needs to be excluded from the transactive dependencies. It would be better just to copy HardLink.java into bookkeeper-server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-185) Remove bookkeeper-server dependency on hadoop-common
[ https://issues.apache.org/jira/browse/BOOKKEEPER-185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated BOOKKEEPER-185: -- Attachment: BOOKKEEPER-185.diff Patch copies the HardLink from hadoop-common 0.23.1. A small modification had to be made for makeShellPath. Remove bookkeeper-server dependency on hadoop-common Key: BOOKKEEPER-185 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-185 Project: Bookkeeper Issue Type: Improvement Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 4.1.0 Attachments: BOOKKEEPER-185.diff bookkeeper-server depends on hadoop-common simply for the use of 1 class, HardLink. This clutters the pom.xml as a lot of stuff needs to be excluded from the transactive dependencies. It would be better just to copy HardLink.java into bookkeeper-server. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-164) Add checksumming for ledger index files
[ https://issues.apache.org/jira/browse/BOOKKEEPER-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13228852#comment-13228852 ] Sijie Guo commented on BOOKKEEPER-164: -- sorry, I haven't finished this one. since it's an improvement to make system more robust, not a blocking issue. I think we can move it to 4.2.0. Add checksumming for ledger index files --- Key: BOOKKEEPER-164 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-164 Project: Bookkeeper Issue Type: Improvement Components: bookkeeper-server Affects Versions: 4.0.0 Reporter: Sijie Guo Fix For: 4.1.0 now bookie ledger index files lacks checksumming to prevent truncation/corruption. if a ledger index file is truncated, the ledger index file still works but responds wrong response when reading last confirmed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
