[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568313#comment-13568313 ] Mridul Muralidharan commented on BOOKKEEPER-312: Oh great ! I saw the failed message from hudson, and was not sure what the status was ... My point was that there is nothing much we can do about the findbugs warnings. Thanks for clarifying Flavio; and thanks to Ivan and Matthieu for the reviews and helpful comments ! > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568296#comment-13568296 ] Flavio Junqueira commented on BOOKKEEPER-312: - Hi Mridul, This jira has actually been resolved already. If there are things you'd like to fix, please create other jiras. Thanks! > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568271#comment-13568271
]
Mridul Muralidharan commented on BOOKKEEPER-312:
I am not sure which is the actual subset which is generated in the report
above, but I ran findbugs explicitly and these are the results I got, and
explainations of the same :
a) Correctness (5).
a.1) Null pointer dereference.
This is mandated by the jms spec weirdly.
Each of the 5 cases have an explicit comment above them which states that an
NPE is to be thrown as part of the operation.
b) Bad practise (63)
b.1) Confusing method names (30)
Javacc generated methods.
b.2) Incorrect definition of Serializable class (2)
Protobuf generated code.
b.3) Method with Boolean return type returns explicit null (31)
This is explicitly used in our code - we make use of a Boolean return value;
and it is interpreted as true, false or null (when there are parse errors, etc).
Non standard, but explicitly written and expected use : note, this is only
within the selector evaluation and highly specific to the function evaluations
within selector : DOES NOT leak to outside the selector package.
c) Internationalization (4)
javacc generated code.
d) Malicious code vulnerability (5)
The subtree of bugs refers to javacc generated code.
e) Performance (2)
One of them is from generated code, the other is unavoidable.
f) Dodgy code (22)
f.1) Dead local store (1)
This is being reported since javac is inlining a final variable : within the
code, the variable is used (and so not dead store) - just than findbugs does
not 'see' it since it operated on bytecode.
f.2) Null pointer dereference (17)
These are all from error reporting code. Example:
if (null == topic) throw new InvalidDestinationException("Illegal destination "
+ topic);
f.3) switch fall through (1)
javacc generated code
f.4) Unread field (2)
javacc generated code.
f.5) useless control flow.
protobuf generated code.
I had done this analysis a while back : but had lost the responses (might be in
one of the bugs/review comments ? not sure).
Since our code has moved a bit, it is good to verify that no new bugs were
introduced anyway !
If there are any other bugs/warnings reported by findbugs which does not match
this list (or generated by some other command - I just ran with default mvn
findbugs:findbugs), please do let me know !
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
>
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568242#comment-13568242 ] Mridul Muralidharan commented on BOOKKEEPER-312: How can I get to the findbug warnings/errors generated in this build ? > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567977#comment-13567977 ] Hudson commented on BOOKKEEPER-312: --- Integrated in bookkeeper-trunk #83 (See [https://builds.apache.org/job/bookkeeper-trunk/83/]) BOOKKEEPER-312: Implementation of JMS provider (mridul via ivank) (Revision 1441108) Result = FAILURE ivank : Files : * /zookeeper/bookkeeper/trunk/CHANGES.txt * /zookeeper/bookkeeper/trunk/hedwig-client-jms * /zookeeper/bookkeeper/trunk/hedwig-client-jms/pom.xml * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/grammar * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/grammar/javacc * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/grammar/javacc/readme.html * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/grammar/javacc/selector_grammar.jjt * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/ConnectionImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/ConnectionMetaDataImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/DebugUtil.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/LRUCacheMap.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/LRUCacheSet.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/MessagingSessionFacade.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/Mutable.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/SessionImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/StateManager.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/jndi * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/jndi/HedwigInitialContext.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/jndi/HedwigInitialContextFactory.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/jndi/package-info.html * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/BytesMessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/MapMessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/MessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/MessageUtil.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/ObjectMessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/StreamMessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/TextMessageImpl.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/header * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/header/MetadataProcessor.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/package-info.html * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/package-info.html * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector/BinaryArithmeticFunction.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector/BinaryExprFunction.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector/ExprFunction.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector/InterpretSelectorParserVisitor.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/apache/hedwig/jms/selector/LogicalComparisonFunction.java * /zookeeper/bookkeeper/trunk/hedwig-client-jms/src/main/java/org/
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567904#comment-13567904
]
Ivan Kelly commented on BOOKKEEPER-312:
---
The remaining trailing spaces and long lines are in JmsHeader.java, which is a
generated file.
{quote}
$ cat
./hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java
| grep -c -e " $"
93
$ cat
./hedwig-client-jms/src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java
| awk 'BEGIN{count=0}{if ( length > 121 ) { count=count+1} }END{ print count}'
6
{quote}
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), then they will be delivered irrespective
> of stop().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567900#comment-13567900
]
Hadoop QA commented on BOOKKEEPER-312:
--
Testing JIRA BOOKKEEPER-312
Patch
[BOOKKEEPER-312.diff|https://issues.apache.org/jira/secure/attachment/12567396/BOOKKEEPER-312.diff]
downloaded at Thu Jan 31 17:50:44 UTC 2013
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:red}-1{color} the patch contains 93 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 6 line(s) longer than 120
characters
.{color:green}+1{color} the patch does adds/modifies 104 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 FINDBUGS{color}
.{color:green}+1{color} the patch does not seem to introduce new Findbugs
warnings
{color:green}+1 TESTS{color}
.Tests run: 816
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/259/
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), t
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567876#comment-13567876 ] Matthieu Morel commented on BOOKKEEPER-312: --- +1 All formatting issues have been taken care of (thanks Ivan!), and limitations for this experimental feature are clearly outlined in this patch. We could include them in a readme file in the jms subproject, but that can be done later. Great job Mridul! > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565644#comment-13565644 ] Mridul Muralidharan commented on BOOKKEEPER-312: Hi Ivan, a) The github repo I have is horribly out of date (and has other changes in it) : as of now, the only thing I have is the patch which finished the review. b) It is not a direct import of the activemq testcode : but it has gone through quite a bit of change to allow use with hedwig (most of it script generated). The only reason we import the activemq code is to allow us to test the corner cases in jms spec. Though it is not mandatory to functionality include it (worst case, we can drop it - just like we dropped JORAM testcases due to incompatible license) : but imo, considering the significant value they bring, we should include it if possible. c) Regarding licenses : that is weird, my silly scripts did not catch them ! I will add the license and re-submit : thanks for the pointer, that was a bad oversight; my apologies > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565603#comment-13565603 ] Ivan Kelly commented on BOOKKEEPER-312: --- Do you have a github repo with your changes in it? I'll see what I can do about trailing space. Is the activemq code the only external code that's been brought in? There's also rat errors. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/256/artifact/test-patch/reports/RAT-patch.txt. There's a list of the files with unapproved licenses about 1/10 of the way down. Note that to run rat locally, you need to have a clean branch, otherwise you'll get a lot of false positives. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565194#comment-13565194 ] Mridul Muralidharan commented on BOOKKEEPER-312: A) About @author tags: The author tags are from activemq testcases. Currently we do very minimal changes to them in the hope that we can do future merges from their codebase (unfortunately, this might not be posible anymore, but still ...) $ find . -type f | xargs grep -i author | grep @ ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/usecases/TransactionRollbackOrderTest.java: * @author Paul Smith ./src/test/java/org/apache/activemq/usecases/DispatchMultipleConsumersTest.java: * @author Rajani Chennamaneni ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/TransactionTest.java: * @author pragmasoft ./src/test/java/org/apache/activemq/usecases/SubscribeClosePublishThenConsumeTest.java: * @author Paul Smith B) About tabs in code: Again, this is coming from activemq codebase : (ctrl+v, tab as grep param below) $ find . -type f | grep -v \.class | xargs grep '' | grep -v activemq Binary file ./target/hedwig-client-jms-4.3.0-SNAPSHOT.jar matches C) trailing spaces There are quite a lot of cases of trailing spaces - including in generated code. Is this mandatory to remove this ? It will be very timeconsuming to do so manually; and I dont have the time to try to script it. If there is something already present, I can ofcourse run it and resubmit patch. D) Files with lines longer than 120 characters. There are a bunch of files with lines longer than 120 characters : mostly in comments and testcases. I will try to remove them if it is a priority - unfortunately, I wont be able to get to these for a while due to other commitments. $ for i in `find src -type f `; do if [ -n "`awk 'length($0) > 120' $i`" ]; then echo $i; fi; done | grep -v activemq | grep java$ src/test/java/org/apache/hedwig/jms/selector/BasicSelectorGrammarTest.java src/test/java/org/apache/hedwig/jms/BasicJMSTest.java src/test/java/org/apache/hedwig/JmsTestBase.java src/main/java/org/apache/hedwig/jms/selector/ValueComparisonFunction.java src/main/java/org/apache/hedwig/jms/SessionImpl.java src/main/java/org/apache/hedwig/jms/ConnectionImpl.java src/main/java/org/apache/hedwig/jms/spi/HedwigMessagingSessionFacade.java src/main/java/org/apache/hedwig/jms/spi/MessageProducerImpl.java src/main/java/org/apache/hedwig/jms/spi/TopicSubscriberImpl.java src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562615#comment-13562615
]
Hadoop QA commented on BOOKKEEPER-312:
--
Testing JIRA BOOKKEEPER-312
Patch
[hedwig-client-jms.patch.10|https://issues.apache.org/jira/secure/attachment/12566500/hedwig-client-jms.patch.10]
downloaded at Fri Jan 25 11:00:40 UTC 2013
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:red}-1{color} the patch seems to contain 6 line(s) with @author tags
.{color:red}-1{color} the patch contains 6 line(s) with tabs
.{color:red}-1{color} the patch contains 444 line(s) with trailing spaces
.{color:red}-1{color} the patch contains 163 line(s) longer than 120
characters
.{color:green}+1{color} the patch does adds/modifies 104 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce 18 new RAT warning(s)
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1 FINDBUGS{color}
.{color:green}+1{color} the patch does not seem to introduce new Findbugs
warnings
{color:green}+1 TESTS{color}
.Tests run: 816
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/bookkeeper-trunk-precommit-build/256/
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.10, hedwig-client-jms.patch.10,
> hedwig-client-jms.patch.2, hedwig-client-jms.patch.3,
> hedwig-client-jms.patch.4, hedwig-client-jms.patch.5,
> hedwig-client-jms.patch.9, hedwig-client-jms.patch.9
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, an
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562604#comment-13562604 ] Mridul Muralidharan commented on BOOKKEEPER-312: Please note that, as before, about 22+ tests will fail when doing mvn install - this is because of the unresolved client bug. The output I got was - "Tests run: 195, Failures: 22, Errors: 1, Skipped: 13" > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562596#comment-13562596
]
Hadoop QA commented on BOOKKEEPER-312:
--
Testing JIRA BOOKKEEPER-312
Patch
[hedwig-client-jms.patch.10|https://issues.apache.org/jira/secure/attachment/12566499/hedwig-client-jms.patch.10]
downloaded at Fri Jan 25 10:50:40 UTC 2013
{color:red}-1{color} Patch failed to apply to head of branch
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.10, hedwig-client-jms.patch.2,
> hedwig-client-jms.patch.3, hedwig-client-jms.patch.4,
> hedwig-client-jms.patch.5, hedwig-client-jms.patch.9,
> hedwig-client-jms.patch.9
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), then they will be delivered irrespective
> of stop().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13561844#comment-13561844 ] Ivan Kelly commented on BOOKKEEPER-312: --- BOOKKEEPER-544 is in now. The patch fails the apache-rat test. There's licenses missing from 10 files. I've submitted the patch, so precommit should pick up any other issues. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551837#comment-13551837 ] Sijie Guo commented on BOOKKEEPER-312: -- is your patch generated based on BOOKKEEPER-544? if so, please don't submit it unless your patch is based on latest trunk. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551830#comment-13551830 ] Mridul Muralidharan commented on BOOKKEEPER-312: $ git apply ~/Desktop/hedwig-client-jms.patch.9 /home/mridulm/Desktop/hedwig-client-jms.patch.9:8075: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8077: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8078: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8080: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8087: trailing whitespace. warning: squelched 476 whitespace errors warning: 481 lines add whitespace errors. $ git status # On branch trunk # Changes not staged for commit: # (use "git add ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: pom.xml # # Untracked files: # (use "git add ..." to include in what will be committed) # # hedwig-client-jms/ no changes added to commit (use "git add" and/or "git commit -a") Which is the expected result of applying this patch. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551635#comment-13551635
]
Hadoop QA commented on BOOKKEEPER-312:
--
Testing JIRA BOOKKEEPER-312
Patch
[hedwig-client-jms.patch.9|https://issues.apache.org/jira/secure/attachment/12564511/hedwig-client-jms.patch.9]
downloaded at Fri Jan 11 23:32:17 UTC 2013
{color:red}-1{color} Patch failed to apply to head of branch
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.2, hedwig-client-jms.patch.3,
> hedwig-client-jms.patch.4, hedwig-client-jms.patch.5,
> hedwig-client-jms.patch.9, hedwig-client-jms.patch.9
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), then they will be delivered irrespective
> of stop().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551587#comment-13551587
]
Hadoop QA commented on BOOKKEEPER-312:
--
Testing JIRA BOOKKEEPER-312
Patch
[hedwig-client-jms.patch.9|https://issues.apache.org/jira/secure/attachment/12564503/hedwig-client-jms.patch.9]
downloaded at Fri Jan 11 22:20:13 UTC 2013
{color:red}-1{color} Patch failed to apply to head of branch
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.2, hedwig-client-jms.patch.3,
> hedwig-client-jms.patch.4, hedwig-client-jms.patch.5,
> hedwig-client-jms.patch.9
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), then they will be delivered irrespective
> of stop().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551558#comment-13551558 ] Mridul Muralidharan commented on BOOKKEEPER-312: Attached final patch which completed the review at https://reviews.apache.org/r/6277/ I had to incorporate a few additional changes, namely : a) Changes to server test case code. b) Need to work around BOOKKEEPER-545. In addition to start delivery, had to also workaround subscribe topic. I think some combination of the two also creates issues. c) dos2unix for some files which had slipped thorugh the review. d) patch generated via - git diff -r c1c8bef6125cb70ed2da68e92bb49e45f4ea64a9 --no-prefix hedwig-client-jms pom.xml >hedwig-client-jms.patch.9 > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528587#comment-13528587 ] Sijie Guo commented on BOOKKEEPER-312: -- marked it for 4.3.0 since its parent jira BOOKKEEPER-308 is marked for 4.3.0. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431745#comment-13431745 ] Sijie Guo commented on BOOKKEEPER-312: -- update other comments in review board: https://reviews.apache.org/r/6277/ > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431729#comment-13431729
]
Sijie Guo commented on BOOKKEEPER-312:
--
@Mridul,
{quote}
There are package info files on each package to give general idea of what it
does.
It will be less overwhelming if you split effort along these lines as opposed
to try to take everything in.
{quote}
thanks for providing these informations.
as I mentioned, splitting the patches would speed up reviewing. If it is
difficult for you to split, I am OK to review, but it might take more time to
finish it :-(
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.2, hedwig-client-jms.patch.3,
> hedwig-client-jms.patch.4, hedwig-client-jms.patch.5
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
> JMS indicates ability to acknowledge individual messages.
> This distinction is currently unsupported.
> 8) JMS spec requires
> "A connection's delivery of incoming messages can be temporarily stopped
> using its stop() method. It can be restarted using its start() method. When
> the connection is stopped, delivery to all the connection’s MessageConsumers
> is inhibited: synchronous receives block, and messages are not delivered to
> MessageListeners."
> We honor this for undelivered messages from server - but if stop is called
> while there are pending messages yet to be delivered to a listener (or
> buffered in subscriber for receive), then they will be delivered irrespective
> of stop().
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428669#comment-13428669 ] Mridul Muralidharan commented on BOOKKEEPER-312: The review board was giving me some issues - I hope all my final comments made it through and not intermediate WIP comments ! A cursory glance did not bring anything up thought. If anything does not make sense, is truncated or is just plain nonsensical, please do let me know - might be more to do with what I wrote and not review board barfing on me but I want to cover all bases :-) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428631#comment-13428631 ] Mridul Muralidharan commented on BOOKKEEPER-312: @Flavio. There are package-info.html files on all packages IIRC - which gives a fairly brief and high level info on what the package does. Admittedly, without referring to JMS spec, a lot of it might not make much sense unfortunately - ofcourse, adding more can help : not sure how much more ! Regarding "JMS Violation" comments. There are two classes of those : a) Where we are forced to deviate from or violate JMS spec due to Hedwig's design choices. b) Where it is not clear (atleast to me) how things should behave. Client api idioms, lack of server side filter support, ack'ing individual messages, etc fall under (a). I have tried to clarify (b) mostly via expected behavior from other test suites and online search - but wanted to call them out in case of future issues. @Sijie Guo I will try to respond to the review request directly - hope that works better than trying here. General comment regarding splitting the patch up - think along package lines - the code is conceptually split along that already. Considering org.apache.hedwig.jms as base : jndi interaction classes under 'jndi', actual hedwig comm will be under 'spi', selector code is under 'selector' (based off grammer under src/main/grammar), message objects under 'message', message metadata under 'message.header' (based off protobuf from src/main/protobuf), and the general jms base code under . There is minimal interaction (except in unfortunate case of . and spi which I could not decouple better due to how code evolved) There are package info files on each package to give general idea of what it does. It will be less overwhelming if you split effort along these lines as opposed to try to take everything in. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undel
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427965#comment-13427965
]
Sijie Guo commented on BOOKKEEPER-312:
--
the overall patch looks great although I just taked a look at the skeleton of
the patch w/o test cases and selector parts.
I had some general comments:
1) code style. it would be better to follow Java code convention for if-else,
while, for blocks.
http://www.oracle.com/technetwork/java/codeconventions-142311.html#449
2) In general, it would be better to provide detail meaningful messages for
exception and log, not just short statement.
3) configurations. it would be better to have a configuration object for it,
even for now we passing settings thru system properties. you could reference
bookkeeper-server/AbstractConfiguration, we used a CompositeConfiguration to
add a SystemConfiguration. so it could read settings from system properties
4) log4j provide mechanism to log the stack like #logger.debug("error message",
exception). you don't need to do it separately using Util.dumpStacks you
provided.
>> comments about Selector.
I think it would be better to define a interface like filter(Message message)
to hide AST related structure under this interface. so we could decouple AST
node (like Node ast ) from main logic. And it would be easy to split the big
patch into several small parts.
>> comments about patch review:
it is a too big patch for reviewing and covering them. My suggestion is to
split it into several smaller patches.
these smaller patches could be:
1) skeleton of the code. publisher part code and test cases.
2) subscriber (consumer) part code w/o filtering and test cases. (it could be
done if we had clean interface for filtering to hide selector part as my above
comment)
3) selector part and filtering and test cases.
4) for queues, since hedwig doesn't support queue. it would be better to no
include any code about queue, just throw UnsupportOperationException for future
extension.
For this four parts, it would be better to have them in separated jiras. the
later one's patch is generated based on the previous one.
I know it introduces additional works for you, but it did help the community to
review your codes, give comments and improve code quality before any commits.
And I could help you doing the split work if the suggestion makes sense for
you.
other comments about code you could check them in review board:
https://reviews.apache.org/r/6277/
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
> Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1,
> hedwig-client-jms.patch.2, hedwig-client-jms.patch.3,
> hedwig-client-jms.patch.4, hedwig-client-jms.patch.5
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427906#comment-13427906 ] Flavio Junqueira commented on BOOKKEEPER-312: - A couple of high level comments before I forget: # It would be great to have documentation # Do comments like this "// JMS VIOLATION: we will be unable to check for propertyExists after this" require a jira? > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427161#comment-13427161 ] Mridul Muralidharan commented on BOOKKEEPER-312: I assumed review board will send this out, but did not see any updates. The review request is : https://reviews.apache.org/r/6277/ Please let me know if there are any issues accessing it. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426239#comment-13426239 ] Sijie Guo commented on BOOKKEEPER-312: -- @Mridul, your patch is generated using git. so please choose bookkeeper-git as your repository. please try again. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426018#comment-13426018 ] Mridul Muralidharan commented on BOOKKEEPER-312: As I mentioned earlier at BOOKKEEPER-308, the review board gives me (when I tried again now) " Something broke! (Error 500) It appears something broke when you tried to go to here. This is either a bug in Review Board or a server configuration error. Please report this to your administrator. " I am using Repository: Bookkeeper. Base Directory: / Diff: (path on my hdd to diff - as attached above) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425865#comment-13425865 ] Sijie Guo commented on BOOKKEEPER-312: -- @Mridul, could you put the patch in review board, since it is a big patch? it is quite difficult to make comments over a big patch. thanks in advance. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414660#comment-13414660 ] Sijie Guo commented on BOOKKEEPER-312: -- could you put the patch in on review board, since it is a big patch. review board: https://reviews.apache.org/ > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
