[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565194#comment-13565194 ] Mridul Muralidharan commented on BOOKKEEPER-312: A) About @author tags: The author tags are from activemq testcases. Currently we do very minimal changes to them in the hope that we can do future merges from their codebase (unfortunately, this might not be posible anymore, but still ...) $ find . -type f | xargs grep -i author | grep @ ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/usecases/TransactionRollbackOrderTest.java: * @author Paul Smith ./src/test/java/org/apache/activemq/usecases/DispatchMultipleConsumersTest.java: * @author Rajani Chennamaneni ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/TransactionTest.java: * @author pragmasoft ./src/test/java/org/apache/activemq/usecases/SubscribeClosePublishThenConsumeTest.java: * @author Paul Smith B) About tabs in code: Again, this is coming from activemq codebase : (ctrl+v, tab as grep param below) $ find . -type f | grep -v \.class | xargs grep '' | grep -v activemq Binary file ./target/hedwig-client-jms-4.3.0-SNAPSHOT.jar matches C) trailing spaces There are quite a lot of cases of trailing spaces - including in generated code. Is this mandatory to remove this ? It will be very timeconsuming to do so manually; and I dont have the time to try to script it. If there is something already present, I can ofcourse run it and resubmit patch. D) Files with lines longer than 120 characters. There are a bunch of files with lines longer than 120 characters : mostly in comments and testcases. I will try to remove them if it is a priority - unfortunately, I wont be able to get to these for a while due to other commitments. $ for i in `find src -type f `; do if [ -n `awk 'length($0) 120' $i` ]; then echo $i; fi; done | grep -v activemq | grep java$ src/test/java/org/apache/hedwig/jms/selector/BasicSelectorGrammarTest.java src/test/java/org/apache/hedwig/jms/BasicJMSTest.java src/test/java/org/apache/hedwig/JmsTestBase.java src/main/java/org/apache/hedwig/jms/selector/ValueComparisonFunction.java src/main/java/org/apache/hedwig/jms/SessionImpl.java src/main/java/org/apache/hedwig/jms/ConnectionImpl.java src/main/java/org/apache/hedwig/jms/spi/HedwigMessagingSessionFacade.java src/main/java/org/apache/hedwig/jms/spi/MessageProducerImpl.java src/main/java/org/apache/hedwig/jms/spi/TopicSubscriberImpl.java src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig
Re: [Discussion] Read/Write semantic for BookKeeper and session fencing
Ivan has already given a pretty good summary of the thread. I just wanted to add that the number of metadata writes has been identified as a problem for Hedwig when hubs crash and the topics of a faulty hub need to be reassigned. Getting new owners for these topics is performed eagerly, which causes a spike of metadata accesses. -Flavio From: Ivan Kelly iv...@apache.org To: bookkeeper-dev@zookeeper.apache.org Sent: Tuesday, January 29, 2013 11:15 AM Subject: Re: [Discussion] Read/Write semantic for BookKeeper and session fencing On Mon, Jan 28, 2013 at 08:49:34AM -0800, Benjamin Reed wrote: this thread is a bit long could someone summarize both the motivation and proposal? The motivation is to reduce the number of metadata accesses required for a hedwig hub to take over a topic. Currently it has to close the previous ledger (2 writes) and open a new one (1 write). This thread is proposing a session fencing mechanism, that would reduce this to 1 write, and allow the hub to continue writing to the same ledger. The proposed mechanism (Sijie, correct me if I'm wrong), is to carry a session id with all read/write requests. During recovery, the hub will first read the metadata, which contains the previous session id, increment this and fence one bookie in each quorum. These bookies, will now no longer access write requests with a session id lower than the one used for fencing. The hub then recovers as far as it can, and then makes a single metadata write. This single write contains the first new entry of the new session, and the new session id. Each reading client will compare the session id to whatever session id a message is supposed to have according to the metadata. If this doesn't match, the message is discarded. -Ivan
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565603#comment-13565603 ] Ivan Kelly commented on BOOKKEEPER-312: --- Do you have a github repo with your changes in it? I'll see what I can do about trailing space. Is the activemq code the only external code that's been brought in? There's also rat errors. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/256/artifact/test-patch/reports/RAT-patch.txt. There's a list of the files with unapproved licenses about 1/10 of the way down. Note that to run rat locally, you need to have a clean branch, otherwise you'll get a lot of false positives. Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-555) Make BookieServer use Netty rather than a custom IO server
[ https://issues.apache.org/jira/browse/BOOKKEEPER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Kelly updated BOOKKEEPER-555: -- Attachment: 0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch Patch is code complete, and should be passing all tests 100%. I have yet to benchmark. Make BookieServer use Netty rather than a custom IO server -- Key: BOOKKEEPER-555 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-555 Project: Bookkeeper Issue Type: Bug Reporter: Ivan Kelly Assignee: Ivan Kelly Fix For: 4.3.0 Attachments: 0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch, 0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch Move from the custom NIO server to netty. This will make it easier to do things like add more server side threads and support SSL. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565644#comment-13565644 ] Mridul Muralidharan commented on BOOKKEEPER-312: Hi Ivan, a) The github repo I have is horribly out of date (and has other changes in it) : as of now, the only thing I have is the patch which finished the review. b) It is not a direct import of the activemq testcode : but it has gone through quite a bit of change to allow use with hedwig (most of it script generated). The only reason we import the activemq code is to allow us to test the corner cases in jms spec. Though it is not mandatory to functionality include it (worst case, we can drop it - just like we dropped JORAM testcases due to incompatible license) : but imo, considering the significant value they bring, we should include it if possible. c) Regarding licenses : that is weird, my silly scripts did not catch them ! I will add the license and re-submit : thanks for the pointer, that was a bad oversight; my apologies Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-555) Make BookieServer use Netty rather than a custom IO server
[
https://issues.apache.org/jira/browse/BOOKKEEPER-555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565647#comment-13565647
]
Hadoop QA commented on BOOKKEEPER-555:
--
Testing JIRA BOOKKEEPER-555
Patch
[0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch|https://issues.apache.org/jira/secure/attachment/12567022/0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch]
downloaded at Tue Jan 29 18:50:40 UTC 2013
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
120
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1 FINDBUGS{color}
.{color:red}-1{color} the patch seems to introduce 2 new Findbugs
warning(s) in module(s) [bookkeeper-server]
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/257/
Make BookieServer use Netty rather than a custom IO server
--
Key: BOOKKEEPER-555
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-555
Project: Bookkeeper
Issue Type: Bug
Reporter: Ivan Kelly
Assignee: Ivan Kelly
Fix For: 4.3.0
Attachments: 0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch,
0002-BOOKKEEPER-555-Netty-Server-for-Bookie.patch
Move from the custom NIO server to netty. This will make it easier to do
things like add more server side threads and support SSL.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.10) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.10) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.2) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.3) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.4) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.5) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.10 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.9) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.10 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.9) Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.10 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.10 Fixes license problems that Ivan pointed out ! Implementation of JMS provider -- Key: BOOKKEEPER-312 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 Project: Bookkeeper Issue Type: Sub-task Reporter: Mridul Muralidharan Assignee: Mridul Muralidharan Fix For: 4.3.0 Attachments: hedwig-client-jms.patch.10 The JMS provider implementation conforming to the 1.1 spec. The limitations as of now are : 1) No support for Queue's : Hedwig currently does not have a notion of JMS queue's for us to leverage. 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of connection -(n)- session -(n)- publisher/subscriber. Each session has a hedwig connection. Currently I am simulating noLocal, but this IS fragile and works for the duration of connection - ONLY until the message id is still in a LRUCache. As mentioned before, this is a kludge, and not a good solution. 3) Note that everything is durable in hedwig - so we do not support NON_PERSISTENT delivery mode. 4) Calling unsubscribe on a durable subscription will fail if it was NOT created in the current session. In hedwig, to unsubscribe, we need the subscription id and the topic ... To simulate unsubscribe(), we store the subscriberId to topicName mapping when a create* api is invoked. Hence, if create* was NOT called, then we have no way to infer which topic the subscription-id refers to from hedwig, and so cant unsubscribe. The workaround is - simply create a durable subsriber just as a workaround of this limitation - the topicName will be known to the user/client anyway. 5) Explicit session recovery is not supported. Reconnection of hedwig session (either explicitly or implicitly by underlying client implementation) will automatically trigger redelivery of un-acknowledged messages. 6) Because of the above, setting the JMSRedelivered flag is almost impossible in a consistent way. Currently, we simulate it for redelivery due to provider side events : rollback of txn, exception in message listener (primarily). At best we can simulate it with a kludge - at risk of potentially running out of resources ... this is being investigated : but unlikely to have a clean fix. 7) Hedwig only supports marking all messages until seq-id as received : while JMS indicates ability to acknowledge individual messages. This distinction is currently unsupported. 8) JMS spec requires A connection's delivery of incoming messages can be temporarily stopped using its stop() method. It can be restarted using its start() method. When the connection is stopped, delivery to all the connection’s MessageConsumers is inhibited: synchronous receives block, and messages are not delivered to MessageListeners. We honor this for undelivered messages from server - but if stop is called while there are pending messages yet to be delivered to a listener (or buffered in subscriber for receive), then they will be delivered irrespective of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
