[jira] [Updated] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-648: --- Assignee: (was: Mridul Muralidharan) > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (BOOKKEEPER-560) Create readme for hedwig-client-jms
[ https://issues.apache.org/jira/browse/BOOKKEEPER-560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-560: --- Assignee: (was: Mridul Muralidharan) > Create readme for hedwig-client-jms > > > Key: BOOKKEEPER-560 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-560 > Project: Bookkeeper > Issue Type: Task > Components: Documentation >Reporter: Ivan Kelly > > This module needs a readme describing it as an experimental component and > what parts of jms are supported and not supported. > It would also be good to have a bit more detail on why they can't be > supported with hedwig as it is today. A lot of this can be taken verbatim > from the package-info.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13720544#comment-13720544 ] Mridul Muralidharan commented on BOOKKEEPER-648: To add to my comment above : The test tries to ensure that, a) all messages sent are received in both sync and async case. b) they are received in the right order. c) there is no data loss (properties, text are checked). Spinning until we receive the message can cause tests to hang in case of failure due to (a). > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13720540#comment-13720540 ] Mridul Muralidharan commented on BOOKKEEPER-648: I was expecting 110+ ms (initial 10ms and then 100ms per message) to be extremely high for receiving a message sent locally, but if it is insufficient, the timeout should be adjusted to something more appropriate. The tradeoff is if it is an actual failure, or bad performance regressions, then the test time will increase propotionally. > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13719348#comment-13719348 ] Mridul Muralidharan commented on BOOKKEEPER-648: Flavio, I am not sure what the expectation of this bug is - is it appropriate debug message for this testcase ? As I detailed above - the only message possible is : "no message was received before timeout" - unfortunately, I dont think that is going to help us much in debugging it. The reason I did not add descriptive message for every assertion in the tests is because the corresponding JMS api's detail the error conditions in detail (when a null can be returned from receive is detailed in JMS api javadocs for example). To actually debug/fix the issue, we will need to enable debug logging in server, client api and jms mode. Subsequently, when issue is observed, we will need to trace whether message was actually sent to server, whether server dispatched to both subscribers, whether client api received it, and whether it was dispatched to jms. The jms module does nothing different from any other api user of hedwig - barring bugs in it ofcourse :-) This is an assertion which is typically not expected to fail unless there is something broken elsewhere which is causing message loss. > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13719324#comment-13719324 ] Mridul Muralidharan commented on BOOKKEEPER-648: On going over the code again, the testcase is extremely simple - it tests the sync and async api. a) Create one publisher and two subscriber sessions for the topic (one via sync api, another async api of jms). b) Send 4 messages through publisher session. c) Sleep for 10 ms to ensure all messages have been sent - this should not be required, but hedwig sometimes takes a lot of time. d) For sync test, wait for 100 ms for each message to be received : null is returned if it times out without receiving any message. The number of times receive is called is equal to the number of times we sent messages - the test expects in-order-delivery without loss of messages as per hedwig api contract. This is what is failing : we are not receiving all the messages we sent. e) The async session listener does the same as (d) - except on the async listener : this did not get tested in this specific case due to validation failure in (d). Looking more, we should probably add a sleep before checking the 'messageCount.getValue() != CHAT_MESSAGES.length' condition - in case async listener is still running in parallel : though this is not the failure we are observing ... Assuming you noticed the same assertion stacktrace in each case when it failed, it means no message was received before timeout in sync invocation. This can be due to : 1) Hedwig or bookkeeper is inordinately slow for some reasons (slow hdd, filled up /tmp, low mem, tlb thrashing, etc ?) : in which case, simply bumping up the sleep time and receive timeout param will circumvent the issue. 2) There is some bug somewhere in the chain which is causing message drops - either at publish time or while sending it to subscribers or somewhere else ? To get additional debugging info, there are log messages in jms module : but (particularly for this testcase) the jms module is a thin wrapper delegating to corresponding hedwig client api - so enabling debug there would be more helpful. Actually, I would validate if the server actually sent messages to both the subscribers and they were received by the client - if yes, rest would be a client side bug (hedwig client or jms). If you could reproduce the issue with debug logging enabled for root logger, I can definitely help narrow down the issue with those logs ! Unfortunately, there are almost no testcases in hedwig client : so I am not sure what design or implementation changes happened in client (or server ?) - since I am not keeping track of bookkeeper/hedwig anymore. > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718369#comment-13718369 ] Mridul Muralidharan commented on BOOKKEEPER-648: I tried it 3 times - a) mvn test b) mvn install Both from the jms subdir and c) mvn install for entire bookkeeper. Was there any other env difference when it failed ? Like hdd space constraint, mem constraint, etc ? The basic test is actually pretty basic - send msg, and receive the same if that fails, everything else should also fail. I did not notice any deprecation warnings, so wondering what has changed ! > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[ https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718365#comment-13718365 ] Mridul Muralidharan commented on BOOKKEEPER-648: I tried it 3 times - a) mvn test b) mvn install Both from the jms subdir and c) mvn install for entire bookkeeper. Was there any other env difference ? Like hdd space constraint, mem constraint, etc ? The basic test is actually pretty basic - send msg, and receive the same On Wed, Jul 24, 2013 at 6:51 PM, Flavio Junqueira (JIRA) > BasicJMSTest failed > --- > > Key: BOOKKEEPER-648 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client >Reporter: Flavio Junqueira >Assignee: Mridul Muralidharan > > While running tests, I got once a failure for this hedwig-client-jms test: > BasicJMSTest. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-648) BasicJMSTest failed
[
https://issues.apache.org/jira/browse/BOOKKEEPER-648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13718267#comment-13718267
]
Mridul Muralidharan commented on BOOKKEEPER-648:
I was not able to reproduce this with latest svn trunk.
{noformat}
$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO]
[INFO] Building hedwig-client-jms 4.3.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- javacc-maven-plugin:2.6:jjtree-javacc (jjtree-javacc) @
hedwig-client-jms ---
[INFO] Skipping - all parsers are up to date
[INFO]
[INFO] --- maven-remote-resources-plugin:1.1:process (default) @
hedwig-client-jms ---
[INFO] Setting property: classpath.resource.loader.class =>
'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @
hedwig-client-jms ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
hedwig-client-jms ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 76 source files to
/home/mridulm/work/bookkeeper/trunk/hedwig-client-jms/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @
hedwig-client-jms ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/home/mridulm/work/bookkeeper/trunk/hedwig-client-jms/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.0:testCompile (default-testCompile) @
hedwig-client-jms ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 104 source files to
/home/mridulm/work/bookkeeper/trunk/hedwig-client-jms/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.9:test (default-test) @ hedwig-client-jms ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-surefire-plugin:2.9:test (unit-tests) @ hedwig-client-jms ---
[INFO] Surefire report directory:
/home/mridulm/work/bookkeeper/trunk/hedwig-client-jms/target/surefire-reports
---
T E S T S
---
---
T E S T S
---
Running org.apache.hedwig.jms.BasicJMSTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.516 sec
Running org.apache.hedwig.jms.selector.activemq.SelectorTest
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.208 sec
Running org.apache.hedwig.jms.selector.activemq.SelectorParserTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.195 sec
Running org.apache.hedwig.jms.selector.BasicSelectorGrammarTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.238 sec
Results :
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0
{noformat}
> BasicJMSTest failed
> ---
>
> Key: BOOKKEEPER-648
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-648
> Project: Bookkeeper
> Issue Type: Bug
> Components: hedwig-client
>Reporter: Flavio Junqueira
>Assignee: Mridul Muralidharan
>
> While running tests, I got once a failure for this hedwig-client-jms test:
> BasicJMSTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568313#comment-13568313 ] Mridul Muralidharan commented on BOOKKEEPER-312: Oh great ! I saw the failed message from hudson, and was not sure what the status was ... My point was that there is nothing much we can do about the findbugs warnings. Thanks for clarifying Flavio; and thanks to Ivan and Matthieu for the reviews and helpful comments ! > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[
https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568271#comment-13568271
]
Mridul Muralidharan commented on BOOKKEEPER-312:
I am not sure which is the actual subset which is generated in the report
above, but I ran findbugs explicitly and these are the results I got, and
explainations of the same :
a) Correctness (5).
a.1) Null pointer dereference.
This is mandated by the jms spec weirdly.
Each of the 5 cases have an explicit comment above them which states that an
NPE is to be thrown as part of the operation.
b) Bad practise (63)
b.1) Confusing method names (30)
Javacc generated methods.
b.2) Incorrect definition of Serializable class (2)
Protobuf generated code.
b.3) Method with Boolean return type returns explicit null (31)
This is explicitly used in our code - we make use of a Boolean return value;
and it is interpreted as true, false or null (when there are parse errors, etc).
Non standard, but explicitly written and expected use : note, this is only
within the selector evaluation and highly specific to the function evaluations
within selector : DOES NOT leak to outside the selector package.
c) Internationalization (4)
javacc generated code.
d) Malicious code vulnerability (5)
The subtree of bugs refers to javacc generated code.
e) Performance (2)
One of them is from generated code, the other is unavoidable.
f) Dodgy code (22)
f.1) Dead local store (1)
This is being reported since javac is inlining a final variable : within the
code, the variable is used (and so not dead store) - just than findbugs does
not 'see' it since it operated on bytecode.
f.2) Null pointer dereference (17)
These are all from error reporting code. Example:
if (null == topic) throw new InvalidDestinationException("Illegal destination "
+ topic);
f.3) switch fall through (1)
javacc generated code
f.4) Unread field (2)
javacc generated code.
f.5) useless control flow.
protobuf generated code.
I had done this analysis a while back : but had lost the responses (might be in
one of the bugs/review comments ? not sure).
Since our code has moved a bit, it is good to verify that no new bugs were
introduced anyway !
If there are any other bugs/warnings reported by findbugs which does not match
this list (or generated by some other command - I just ran with default mvn
findbugs:findbugs), please do let me know !
> Implementation of JMS provider
> --
>
> Key: BOOKKEEPER-312
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312
> Project: Bookkeeper
> Issue Type: Sub-task
>Reporter: Mridul Muralidharan
>Assignee: Mridul Muralidharan
> Fix For: 4.3.0
>
> Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10
>
>
> The JMS provider implementation conforming to the 1.1 spec.
> The limitations as of now are :
> 1) No support for Queue's : Hedwig currently does not have a notion of JMS
> queue's for us to leverage.
> 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of
> connection -(n)-> session -(n)-> publisher/subscriber. Each session has a
> hedwig connection.
> Currently I am simulating noLocal, but this IS fragile and works for the
> duration of connection - ONLY until the message id is still in a LRUCache. As
> mentioned before, this is a kludge, and not a good solution.
> 3) Note that everything is durable in hedwig - so we do not support
> NON_PERSISTENT delivery mode.
> 4) Calling unsubscribe on a durable subscription will fail if it was NOT
> created in the current session.
> In hedwig, to unsubscribe, we need the subscription id and the topic ...
> To simulate unsubscribe(), we store the subscriberId to topicName mapping
> when a create* api is invoked. Hence, if create* was NOT called, then we have
> no way to infer which topic the subscription-id refers to from hedwig, and so
> cant unsubscribe.
> The workaround is - simply create a durable subsriber just as a workaround of
> this limitation - the topicName will be known to the user/client anyway.
> 5) Explicit session recovery is not supported.
> Reconnection of hedwig session (either explicitly or implicitly by underlying
> client implementation) will automatically trigger redelivery of
> un-acknowledged messages.
> 6) Because of the above, setting the JMSRedelivered flag is almost impossible
> in a consistent way.
> Currently, we simulate it for redelivery due to provider side events :
> rollback of txn, exception in message listener (primarily).
> At best we can simulate it with a kludge - at risk of potentially running out
> of resources ... this is being investigated : but unlikely to have a clean
> fix.
> 7) Hedwig only supports marking all messages until seq-id as received : while
>
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568242#comment-13568242 ] Mridul Muralidharan commented on BOOKKEEPER-312: How can I get to the findbug warnings/errors generated in this build ? > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: BOOKKEEPER-312.diff, hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.9) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.10 Fixes license problems that Ivan pointed out ! > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.9) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.5) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.10 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.4) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, > hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.2) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.3) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.10) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.10) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.1) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565644#comment-13565644 ] Mridul Muralidharan commented on BOOKKEEPER-312: Hi Ivan, a) The github repo I have is horribly out of date (and has other changes in it) : as of now, the only thing I have is the patch which finished the review. b) It is not a direct import of the activemq testcode : but it has gone through quite a bit of change to allow use with hedwig (most of it script generated). The only reason we import the activemq code is to allow us to test the corner cases in jms spec. Though it is not mandatory to functionality include it (worst case, we can drop it - just like we dropped JORAM testcases due to incompatible license) : but imo, considering the significant value they bring, we should include it if possible. c) Regarding licenses : that is weird, my silly scripts did not catch them ! I will add the license and re-submit : thanks for the pointer, that was a bad oversight; my apologies > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13565194#comment-13565194 ] Mridul Muralidharan commented on BOOKKEEPER-312: A) About @author tags: The author tags are from activemq testcases. Currently we do very minimal changes to them in the hope that we can do future merges from their codebase (unfortunately, this might not be posible anymore, but still ...) $ find . -type f | xargs grep -i author | grep @ ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/perf/NetworkedSyncTest.java: * @author tmielke ./src/test/java/org/apache/activemq/usecases/TransactionRollbackOrderTest.java: * @author Paul Smith ./src/test/java/org/apache/activemq/usecases/DispatchMultipleConsumersTest.java: * @author Rajani Chennamaneni ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/DiscriminatingConsumerLoadTest.java: * @author jlyons ./src/test/java/org/apache/activemq/usecases/TransactionTest.java: * @author pragmasoft ./src/test/java/org/apache/activemq/usecases/SubscribeClosePublishThenConsumeTest.java: * @author Paul Smith B) About tabs in code: Again, this is coming from activemq codebase : (ctrl+v, tab as grep param below) $ find . -type f | grep -v \.class | xargs grep '' | grep -v activemq Binary file ./target/hedwig-client-jms-4.3.0-SNAPSHOT.jar matches C) trailing spaces There are quite a lot of cases of trailing spaces - including in generated code. Is this mandatory to remove this ? It will be very timeconsuming to do so manually; and I dont have the time to try to script it. If there is something already present, I can ofcourse run it and resubmit patch. D) Files with lines longer than 120 characters. There are a bunch of files with lines longer than 120 characters : mostly in comments and testcases. I will try to remove them if it is a priority - unfortunately, I wont be able to get to these for a while due to other commitments. $ for i in `find src -type f `; do if [ -n "`awk 'length($0) > 120' $i`" ]; then echo $i; fi; done | grep -v activemq | grep java$ src/test/java/org/apache/hedwig/jms/selector/BasicSelectorGrammarTest.java src/test/java/org/apache/hedwig/jms/BasicJMSTest.java src/test/java/org/apache/hedwig/JmsTestBase.java src/main/java/org/apache/hedwig/jms/selector/ValueComparisonFunction.java src/main/java/org/apache/hedwig/jms/SessionImpl.java src/main/java/org/apache/hedwig/jms/ConnectionImpl.java src/main/java/org/apache/hedwig/jms/spi/HedwigMessagingSessionFacade.java src/main/java/org/apache/hedwig/jms/spi/MessageProducerImpl.java src/main/java/org/apache/hedwig/jms/spi/TopicSubscriberImpl.java src/main/java/org/apache/hedwig/jms/message/header/JmsHeader.java > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562604#comment-13562604 ] Mridul Muralidharan commented on BOOKKEEPER-312: Please note that, as before, about 22+ tests will fail when doing mvn install - this is because of the unresolved client bug. The output I got was - "Tests run: 195, Failures: 22, Errors: 1, Skipped: 13" > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.10 Trying with : git diff --no-prefix -r 582370359c8e0e399b30df5a7e60a569b0d5aadf pom.xml hedwig-client-jms > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.10, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.10 The issue seems to be version hardcoded into the pom.xml : should be 4.3.0 instead of 4.2.0 (which was valid before 4.3 was released :-) ). Changed that and added patch. Regarding missing license, I tried this : $ for i in `find hedwig-client-jms -type f`; do if [ -z "`grep -v 'Apache Software Foundation' $i`" ]; then echo $i; fi; done Not a foolproof way to detect missing license, but I did not find any files missing. Where can I get the list of files detected with missing license ? > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.10, hedwig-client-jms.patch.2, > hedwig-client-jms.patch.3, hedwig-client-jms.patch.4, > hedwig-client-jms.patch.5, hedwig-client-jms.patch.9, > hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch.1) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch.2) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch.2 Try to workaround the warnings generated > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch.2) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch.2) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch.2 All @RunWith tests need to have public setUp and tearDown methods (weirdness in junit) - ensure that all test classes in hedwig-server conform to this. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch, server_test.patch.1, > server_test.patch.2, server_test.patch.2, server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551830#comment-13551830 ] Mridul Muralidharan commented on BOOKKEEPER-312: $ git apply ~/Desktop/hedwig-client-jms.patch.9 /home/mridulm/Desktop/hedwig-client-jms.patch.9:8075: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8077: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8078: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8080: trailing whitespace. /home/mridulm/Desktop/hedwig-client-jms.patch.9:8087: trailing whitespace. warning: squelched 476 whitespace errors warning: 481 lines add whitespace errors. $ git status # On branch trunk # Changes not staged for commit: # (use "git add ..." to update what will be committed) # (use "git checkout -- ..." to discard changes in working directory) # # modified: pom.xml # # Untracked files: # (use "git add ..." to include in what will be committed) # # hedwig-client-jms/ no changes added to commit (use "git add" and/or "git commit -a") Which is the expected result of applying this patch. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch.2 compile errors ? unexpected - retrying ... > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch, server_test.patch.1, > server_test.patch.2, server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.9 Generated via git diff -r a0ae97c232caafa1fda9955f9aaa9f99564b74fe hedwig-client-jms pom.xml >hedwig-client-jms.patch.9 > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9, hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch.2 Verified to work against jms client ported to trunk. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch, server_test.patch.1, > server_test.patch.2 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.9 > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551558#comment-13551558 ] Mridul Muralidharan commented on BOOKKEEPER-312: Attached final patch which completed the review at https://reviews.apache.org/r/6277/ I had to incorporate a few additional changes, namely : a) Changes to server test case code. b) Need to work around BOOKKEEPER-545. In addition to start delivery, had to also workaround subscribe topic. I think some combination of the two also creates issues. c) dos2unix for some files which had slipped thorugh the review. d) patch generated via - git diff -r c1c8bef6125cb70ed2da68e92bb49e45f4ea64a9 --no-prefix hedwig-client-jms pom.xml >hedwig-client-jms.patch.9 > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: (was: hedwig-client-jms.patch.9) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.9 > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Fix For: 4.3.0 > > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5, > hedwig-client-jms.patch.9 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-545) Regression in subscriber.startDelivery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13551522#comment-13551522 ] Mridul Muralidharan commented on BOOKKEEPER-545: Attached testcase showcases the problem. In a nutshell - in earlier version(s), trying to start delivery on the same topic + subscriberid for a hedwig client multiple times would cause a benign AlreadyStartDeliveryException. This is on longer the case : and the connection is abruptly closed. > Regression in subscriber.startDelivery > -- > > Key: BOOKKEEPER-545 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-545 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client, hedwig-server > Environment: All >Reporter: Mridul Muralidharan >Priority: Critical > Attachments: TestClient.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-545) Regression in subscriber.startDelivery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-545: --- Attachment: TestClient.java > Regression in subscriber.startDelivery > -- > > Key: BOOKKEEPER-545 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-545 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client, hedwig-server > Environment: All >Reporter: Mridul Muralidharan >Priority: Critical > Attachments: TestClient.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-545) Regression in subscriber.startDelivery
[ https://issues.apache.org/jira/browse/BOOKKEEPER-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-545: --- Priority: Major (was: Critical) > Regression in subscriber.startDelivery > -- > > Key: BOOKKEEPER-545 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-545 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-client, hedwig-server > Environment: All >Reporter: Mridul Muralidharan > Attachments: TestClient.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (BOOKKEEPER-545) Regression in subscriber.startDelivery
Mridul Muralidharan created BOOKKEEPER-545: -- Summary: Regression in subscriber.startDelivery Key: BOOKKEEPER-545 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-545 Project: Bookkeeper Issue Type: Bug Components: hedwig-client, hedwig-server Environment: All Reporter: Mridul Muralidharan Priority: Critical -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch.1 setUp and tearDown methods are expected to be protected by convention - changing them to public breaks downstream classes which depend on this method signature (specifically, activemq tests we have) : updating the method signatures to be inline with junit conventions. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch, server_test.patch.1 > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch Modifies the server test code to allow client testcases to leverage the ability to bring up/shutdown a hedwig server to runs tests against (without needing one to be already up). This is required by the jms testcases. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: (was: server_test.patch) > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Attachment: server_test.patch Modifies the server test code to allow client testcases to leverage the ability to bring up/shutdown a hedwig server to runs tests against (without needing one to be already up). This is required by the jms testcases. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > Attachments: server_test.patch > > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned BOOKKEEPER-544: -- Assignee: Mridul Muralidharan > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
[ https://issues.apache.org/jira/browse/BOOKKEEPER-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-544: --- Description: Allow client testcases to start/stop hedwig server as part of their setup/teardown. > Modify hedwig server tests to allow client testcases to start/stop them as > part of their tests > -- > > Key: BOOKKEEPER-544 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan >Priority: Minor > > Allow client testcases to start/stop hedwig server as part of their > setup/teardown. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (BOOKKEEPER-544) Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
Mridul Muralidharan created BOOKKEEPER-544: -- Summary: Modify hedwig server tests to allow client testcases to start/stop them as part of their tests Key: BOOKKEEPER-544 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-544 Project: Bookkeeper Issue Type: Bug Components: hedwig-server Reporter: Mridul Muralidharan Priority: Minor -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-308) Add support for JMS implementation for hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447411#comment-13447411 ] Mridul Muralidharan commented on BOOKKEEPER-308: Any updates on this JIRA ? Is it pending on some input from me ? Last I checked, the review board was updated with the review comments ... Since I am working on other projects - I dont want to miss updates and/or block the issue on something which is pending on me ! > Add support for JMS implementation for hedwig > - > > Key: BOOKKEEPER-308 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-308 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Mridul Muralidharan > > Umbrella bug to keep track of all constituent bugs related to the JMS > provider implementation for hedwig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428669#comment-13428669 ] Mridul Muralidharan commented on BOOKKEEPER-312: The review board was giving me some issues - I hope all my final comments made it through and not intermediate WIP comments ! A cursory glance did not bring anything up thought. If anything does not make sense, is truncated or is just plain nonsensical, please do let me know - might be more to do with what I wrote and not review board barfing on me but I want to cover all bases :-) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13428631#comment-13428631 ] Mridul Muralidharan commented on BOOKKEEPER-312: @Flavio. There are package-info.html files on all packages IIRC - which gives a fairly brief and high level info on what the package does. Admittedly, without referring to JMS spec, a lot of it might not make much sense unfortunately - ofcourse, adding more can help : not sure how much more ! Regarding "JMS Violation" comments. There are two classes of those : a) Where we are forced to deviate from or violate JMS spec due to Hedwig's design choices. b) Where it is not clear (atleast to me) how things should behave. Client api idioms, lack of server side filter support, ack'ing individual messages, etc fall under (a). I have tried to clarify (b) mostly via expected behavior from other test suites and online search - but wanted to call them out in case of future issues. @Sijie Guo I will try to respond to the review request directly - hope that works better than trying here. General comment regarding splitting the patch up - think along package lines - the code is conceptually split along that already. Considering org.apache.hedwig.jms as base : jndi interaction classes under 'jndi', actual hedwig comm will be under 'spi', selector code is under 'selector' (based off grammer under src/main/grammar), message objects under 'message', message metadata under 'message.header' (based off protobuf from src/main/protobuf), and the general jms base code under . There is minimal interaction (except in unfortunate case of . and spi which I could not decouple better due to how code evolved) There are package info files on each package to give general idea of what it does. It will be less overwhelming if you split effort along these lines as opposed to try to take everything in. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undel
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427161#comment-13427161 ] Mridul Muralidharan commented on BOOKKEEPER-312: I assumed review board will send this out, but did not see any updates. The review request is : https://reviews.apache.org/r/6277/ Please let me know if there are any issues accessing it. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426018#comment-13426018 ] Mridul Muralidharan commented on BOOKKEEPER-312: As I mentioned earlier at BOOKKEEPER-308, the review board gives me (when I tried again now) " Something broke! (Error 500) It appears something broke when you tried to go to here. This is either a bug in Review Board or a server configuration error. Please report this to your administrator. " I am using Repository: Bookkeeper. Base Directory: / Diff: (path on my hdd to diff - as attached above) > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424784#comment-13424784 ] Mridul Muralidharan commented on BOOKKEEPER-311: Could be added as pre-commit hook to validate coding conventions and reject if it violates it for java/cpp files ... Ditto for number of warnings, etc. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3, > hedwig-client.patch.4, hedwig-client.patch.5 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424774#comment-13424774 ] Mridul Muralidharan commented on BOOKKEEPER-311: The line length was what was mentioned by Sijie : 80-100 chars per line ... Since the 80 character limit in java coding std predates modern editors (for a time of tty's/consoles using vi/emacs/printed code), I assumed breaking it was fine. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3, > hedwig-client.patch.4, hedwig-client.patch.5 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.5 Changed line length to roughly 100 chars or so. Note that generated code and testcases from activemq are NOT modified. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4, hedwig-client-jms.patch.5 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-311: --- Attachment: hedwig-client.patch.5 Changed line length to roughly 100 chars or so. Note that change is limited to what modified in the patch (not to rest of codebase). > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan >Assignee: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3, > hedwig-client.patch.4, hedwig-client.patch.5 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-330) System.currentTimeMillis usage in Hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13422062#comment-13422062 ] Mridul Muralidharan commented on BOOKKEEPER-330: No, my point was changing from one api to another in the assumption that the bug will be resolved is misplaced - the same issues will recur (in a more unpredictable manner) with nanotime too ... > System.currentTimeMillis usage in Hedwig > > > Key: BOOKKEEPER-330 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-330 > Project: Bookkeeper > Issue Type: Bug > Components: hedwig-server >Affects Versions: 4.0.0, 4.1.0 >Reporter: Sijie Guo >Assignee: Uma Maheswara Rao G > Fix For: 4.2.0 > > Attachments: BOOKKEEPER-330.patch > > > Need same changes in hedwig server as what did in bookkeeper as > BOOKKEEPER-327. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420587#comment-13420587 ] Mridul Muralidharan commented on BOOKKEEPER-311: Sijie Guo has hopefully answered about the JIRA's bq. FORMAT: There are some long lines in this patch, in particular the javadoc for asyncPublishWithResponse. Just to confirm you are referring to the @link tag in javadoc of asyncPublishWithResponse ? If yes : it was autogenerated, let me try to remove the fully qualified names and see if that works. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3, > hedwig-client.patch.4 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-350) Revisit consume interface in Hedwig Client
[ https://issues.apache.org/jira/browse/BOOKKEEPER-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420582#comment-13420582 ] Mridul Muralidharan commented on BOOKKEEPER-350: Please note that this JIRA does not block the JMS provider specifically - it blocks all use of consume api. The JMS provider can be committed without having to resolve this. > Revisit consume interface in Hedwig Client > -- > > Key: BOOKKEEPER-350 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-350 > Project: Bookkeeper > Issue Type: Sub-task > Components: hedwig-client >Affects Versions: 4.0.0, 4.1.0 >Reporter: Sijie Guo > Fix For: 4.2.0 > > > the jira is used to revisit consume interface in hedwig client and to improve > it to meet JMS provider's requirements. > move comments from BOOKKEEPER-311 to here, which make discussion more > clearer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-330) System.currentTimeMillis usage in Hedwig
[
https://issues.apache.org/jira/browse/BOOKKEEPER-330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417686#comment-13417686
]
Mridul Muralidharan commented on BOOKKEEPER-330:
I should have raised the note in BOOKKEEPER-327.
In multi-core (and/or) multi-processor machines (particularly which are running
for long) this method is practically useless for fine measurement.
The precision ('fine') depends on a variety of factors I guess.
System.currentTimeMillis() is at lower resolution (and with time jumps, via ntp
updates, clock skew, etc) - but the measured difference would be what the
system 'thought' the time difference was : should average out to the same
(unless the nodes clock is chronically loosing time).
> System.currentTimeMillis usage in Hedwig
>
>
> Key: BOOKKEEPER-330
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-330
> Project: Bookkeeper
> Issue Type: Bug
> Components: hedwig-server
>Affects Versions: 4.0.0, 4.1.0
>Reporter: Sijie Guo
>Assignee: Uma Maheswara Rao G
> Fix For: 4.2.0
>
> Attachments: BOOKKEEPER-330.patch
>
>
> Need same changes in hedwig server as what did in bookkeeper as
> BOOKKEEPER-327.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417641#comment-13417641 ] Mridul Muralidharan commented on BOOKKEEPER-309: @Flavio I assume you are referring to protobuf generated code ? Is there something specific we can do to get protobuf to generate code conforming to the conventions ? Or am I missing something else ? > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3, hedwig-protocol.patch.4, hedwig-protocol.patch.5 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416130#comment-13416130 ] Mridul Muralidharan commented on BOOKKEEPER-310: Looks good, thanks ! > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: BOOKKEEPER-310.diff, hedwig-server.patch, > hedwig-server.patch.1, hedwig-server.patch.3, hedwig-server.patch.4 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-309: --- Attachment: hedwig-protocol.patch.5 Regenerated with 2.4.1 protobuffer > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3, hedwig-protocol.patch.4, hedwig-protocol.patch.5 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414979#comment-13414979 ] Mridul Muralidharan commented on BOOKKEEPER-309: oh damn, forgot about protoc version ... sigh. Let me regenerate this patch with diffs based on 2.4.1 - thanks ! > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3, hedwig-protocol.patch.4 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414974#comment-13414974 ] Mridul Muralidharan commented on BOOKKEEPER-310: Assuming single server hedwig deployment, this testcase should be sufficient. Thanks ! > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1, > hedwig-server.patch.3, hedwig-server.patch.4 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414948#comment-13414948 ] Mridul Muralidharan commented on BOOKKEEPER-310: The testcases you mention are already present under activemq's testcases under jms provider : I think I already specified this earlier. The caveat is that it wont work in a clustered deployment due to changes in seq-id in remote server (if I understood it right) - the jms testcase does not depend on this. I am not sure what codepaths this testcase tests (or is extended to test), but did not want to make changes which will limit its usability. If that is not the case, we can add some rudimentary checks on the seq-id. > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1, > hedwig-server.patch.3, hedwig-server.patch.4 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-308) Add support for JMS implementation for hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414754#comment-13414754 ] Mridul Muralidharan commented on BOOKKEEPER-308: Btw, I continued with attaching the patches to the JIRA's since review board kept giving me 500 error. > Add support for JMS implementation for hedwig > - > > Key: BOOKKEEPER-308 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-308 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Mridul Muralidharan > > Umbrella bug to keep track of all constituent bugs related to the JMS > provider implementation for hedwig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414752#comment-13414752 ] Mridul Muralidharan commented on BOOKKEEPER-309: Also, had to modify protobuf-java back from 2.4.1 to 2.3.0. I was unable to compile with 2.4.1 inspite of using protobuf lifecycle - the generated code gave compile errors ... > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3, hedwig-protocol.patch.4 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-308) Add support for JMS implementation for hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414750#comment-13414750 ] Mridul Muralidharan commented on BOOKKEEPER-308: Updated with all review comments. Note that there is a dependency on BOOKKEEPER-78 for this to work. > Add support for JMS implementation for hedwig > - > > Key: BOOKKEEPER-308 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-308 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Mridul Muralidharan > > Umbrella bug to keep track of all constituent bugs related to the JMS > provider implementation for hedwig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.4 Add changes to root pom.xml into diff. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3, > hedwig-client-jms.patch.4 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-311: --- Attachment: hedwig-client.patch.4 Based on review comment, add "respBody.hasPublishResponse()" before returning respBody.getPublishResponse() > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3, > hedwig-client.patch.4 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-309: --- Attachment: hedwig-protocol.patch.4 Incorporate review comments - specifically : a) Move to latest apache bookkeeper trunk rev b) remove change(s) from BOOKKEEPER-78 - the rest of the patch DEPENDS on this jira patch as of now to compile and run. > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3, hedwig-protocol.patch.4 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-310: --- Attachment: hedwig-server.patch.4 Incorporate review comments - specifically : a) Move to latest apache bookkeeper trunk rev b) add test case for publish to check if seq-id is returned. c) Fix TestBookKeeperPersistenceManager.java to use modified generic type. > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1, > hedwig-server.patch.3, hedwig-server.patch.4 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-331) Let hedwig support returning message seq id for publish requests.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414647#comment-13414647 ] Mridul Muralidharan commented on BOOKKEEPER-331: bq. 1) all patches need to rebase to latest trunk. a) applying hedwig-protocol.patch.3 failed on PubSubProtocol.java; b) applying hedwig-server.patch.3. it failed to compile test cases due to you don't change TestCallback to return message id in TestBookKeeperPersistenceManager (which is a new test case added in BOOKKEEPER-302). Will do, I will try to regenerate tonight or latest by tomorrow. bq. 2) need a test case to test returning message seq id. you could leverage TestPubSubClient in hedwig-server module. those test cases could be publish & asyncPublish several messages and checking the messages id returned by publish and asyncPublish equals to the message seq id received by a subscriber. This slipped my mind, will add to the testcase - thanks for the pointer. bq. 3) hedwig-protocol.patch : why not use the patch in BOOKKEEPER-78 instead of combining it together? so it could make each jira focus on its changes. Since message headers are a pre-requisite for this set of JIRA's, we can commit them only after the headers are supported (either via this or BOOKKEEPER-78) Do you want me to hold off until BOOKKEEPER-78 is committed ? bq. it would be better to add 'TODO' on the above comment. Yes, I will add. bq. since PublishReponse is an optional filed in ResponseBody, do we need to check whether ResponseBody#hasPublishResponse() before #getPublishResponse() ? Yes, should check and return null if absent. I was assuming protobuf does the same, but it makes sense to be explicit. > Let hedwig support returning message seq id for publish requests. > - > > Key: BOOKKEEPER-331 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-331 > Project: Bookkeeper > Issue Type: Sub-task > Components: hedwig-client, hedwig-server >Reporter: Sijie Guo > Fix For: 4.2.0 > > > Let hedwig support returning message seq id for published messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-311: --- Attachment: (was: hedwig-client.patch.3) > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-311: --- Attachment: hedwig-client.patch.3 > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413181#comment-13413181 ] Mridul Muralidharan commented on BOOKKEEPER-310: Please note that the protocol change is in BOOKKEEPER-309 along with metadata changes. The actual protocol changes are what are detailed in the review comments - so nothing else new there. > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1, > hedwig-server.patch.3 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-331) Let hedwig support returning message seq id for publish requests.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413171#comment-13413171 ] Mridul Muralidharan commented on BOOKKEEPER-331: As of now, post removing all optional changes and bug fixes, this bug is essentially BOOKKEEPER-310 + BOOKKEEPER-311. Did not close the other two issues in favour of this due to the comment trail already present there : but for all intents and purposes, this bug is resolved using the above two jira's mentioned taken together. > Let hedwig support returning message seq id for publish requests. > - > > Key: BOOKKEEPER-331 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-331 > Project: Bookkeeper > Issue Type: Sub-task > Components: hedwig-client, hedwig-server >Reporter: Sijie Guo > Fix For: 4.2.0 > > > Let hedwig support returning message seq id for published messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-308) Add support for JMS implementation for hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413134#comment-13413134 ] Mridul Muralidharan commented on BOOKKEEPER-308: I have removed all changes which are not absolutely mandatory for the implementation to compile and run (consume changes, etc for example). Once this is patched into trunk, we can look into resolving bugs as and when required. All the review comments until now, iirc, are integrated. Please let me know if there are anything left. I am trying to ensure that this gets committed asap due to time pressure. > Add support for JMS implementation for hedwig > - > > Key: BOOKKEEPER-308 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-308 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Mridul Muralidharan > > Umbrella bug to keep track of all constituent bugs related to the JMS > provider implementation for hedwig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-309: --- Attachment: hedwig-protocol.patch.3 Changes to protocol to return seq-id as part of publish response as per review comments. Also includes protocol change from BOOKKEEPER-78 for defining message headers. > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1, > hedwig-protocol.patch.3 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-310: --- Attachment: hedwig-server.patch.3 Removed all spurious and non essential changes to server. The patch now contains only changes to return seq-id for published messages. > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1, > hedwig-server.patch.3 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-311: --- Attachment: hedwig-client.patch.3 Removed all optional changes for base functionality. The patch now only contains changes for retrieving seq-id of a published message, and nothing else. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch, hedwig-client.patch.3 > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-312) Implementation of JMS provider
[ https://issues.apache.org/jira/browse/BOOKKEEPER-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-312: --- Attachment: hedwig-client-jms.patch.3 Use custom header definition based on JIRA feedback. This will, unfortunately, mean that headers used in JMS clients will not be interoperable with other hedwig clients. > Implementation of JMS provider > -- > > Key: BOOKKEEPER-312 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-312 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-jms.patch, hedwig-client-jms.patch.1, > hedwig-client-jms.patch.2, hedwig-client-jms.patch.3 > > > The JMS provider implementation conforming to the 1.1 spec. > The limitations as of now are : > 1) No support for Queue's : Hedwig currently does not have a notion of JMS > queue's for us to leverage. > 2) No support for noLocal : Hedwig DOES NOT conform to JMS model of > connection -(n)-> session -(n)-> publisher/subscriber. Each session has a > hedwig connection. > Currently I am simulating noLocal, but this IS fragile and works for the > duration of connection - ONLY until the message id is still in a LRUCache. As > mentioned before, this is a kludge, and not a good solution. > 3) Note that everything is durable in hedwig - so we do not support > NON_PERSISTENT delivery mode. > 4) Calling unsubscribe on a durable subscription will fail if it was NOT > created in the current session. > In hedwig, to unsubscribe, we need the subscription id and the topic ... > To simulate unsubscribe(), we store the subscriberId to topicName mapping > when a create* api is invoked. Hence, if create* was NOT called, then we have > no way to infer which topic the subscription-id refers to from hedwig, and so > cant unsubscribe. > The workaround is - simply create a durable subsriber just as a workaround of > this limitation - the topicName will be known to the user/client anyway. > 5) Explicit session recovery is not supported. > Reconnection of hedwig session (either explicitly or implicitly by underlying > client implementation) will automatically trigger redelivery of > un-acknowledged messages. > 6) Because of the above, setting the JMSRedelivered flag is almost impossible > in a consistent way. > Currently, we simulate it for redelivery due to provider side events : > rollback of txn, exception in message listener (primarily). > At best we can simulate it with a kludge - at risk of potentially running out > of resources ... this is being investigated : but unlikely to have a clean > fix. > 7) Hedwig only supports marking all messages until seq-id as received : while > JMS indicates ability to acknowledge individual messages. > This distinction is currently unsupported. > 8) JMS spec requires > "A connection's delivery of incoming messages can be temporarily stopped > using its stop() method. It can be restarted using its start() method. When > the connection is stopped, delivery to all the connection’s MessageConsumers > is inhibited: synchronous receives block, and messages are not delivered to > MessageListeners." > We honor this for undelivered messages from server - but if stop is called > while there are pending messages yet to be delivered to a listener (or > buffered in subscriber for receive), then they will be delivered irrespective > of stop(). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409517#comment-13409517 ] Mridul Muralidharan commented on BOOKKEEPER-311: Please note that the guarantees provided must be consistent across api methods exposed - for example, similar reasoning applies to publish. (1) is not supported in publish iirc. Currently, only (2) is. (3) can be inferred if the seq-id is returned : but there is no requirement that not receiving it meant the message was not published (socket lost post delivery or server death, etc) - as in, no transactional guarantees. Which was the reason (and not to mention minimize hedwig changes :) ), we restricted to (2) - make it inline with rest of api, while providing reasonable assurance of delivery. Having said that, as long as there is reasonable assurance that best case effort was made to send consume request to server, any additional guarantees would be better (but would have a higher cost, which needs to be factored in - ack from server for example) ! > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-78) filterable metadata fields in Hedwig's message definition
[ https://issues.apache.org/jira/browse/BOOKKEEPER-78?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409511#comment-13409511 ] Mridul Muralidharan commented on BOOKKEEPER-78: --- I am not sure if I follow, but for server side filtering, the filter (class) specified should be available at the server ? So changing/adding a filter requires (admin) access to and (potentially ?) restarting of the server deployment ? If yes, I can see how it wont be very useful for most production deployments. I hope I am missing something ! > filterable metadata fields in Hedwig's message definition > - > > Key: BOOKKEEPER-78 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-78 > Project: Bookkeeper > Issue Type: New Feature > Components: hedwig-client, hedwig-server >Reporter: Matthieu Morel > > In order to efficiently implement filtering of Hedwig messages, Hedwig should > be able to rely on metadata information. (i.e. without needing to deserialize > the content of the message) > Filtering could use a subset of SQL (like in the JMS spec), leading to > queries such as : > "header1 like 'a' AND header2 IS NOT NULL" > For that purpose, I propose to add customizable metadata to the definition of > Hedwig messages, as header fields. > Metadata must be customizable because it may be arbitrary. We should provide > "map-like" containers according to the type of the metadata field. Metadata > fields would be accessed by name. > There are predefined headers for JMS that could be added as metadata fields > such as : destination (~topic), delivery mode (persistent or not), > expiration, priority, timestamp, correlation id (link to other message), > reply to, type and redelivered. I think only a subset of these should be > predefined headers, if any. > Adding metadata fields to Hedwig messages implies modifying the message > definition, which does not break backward compatibility when those fields are > added as optional in the protocol buffer definition. > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409332#comment-13409332 ] Mridul Muralidharan commented on BOOKKEEPER-309: bq. 1) first is to encode jms message in the body of hedwig message. and mark the hedwig message type as 'JMS' in messageType field in MessageHeader. If I understand this correctly, it will mean that it will not be possible for publisher's using jms provider to interact with hedwig clients (and vice versa) right ? As in, clients which depends only on body will now see body plus metadata encoded within ? If that is the case, then I dont think we should pursue it ... Or did I miss something ? bq. 2) second is to leverage hedwig generic properties field in MessageHeader. you could have JmsValue type in jms.protocol in hedwig-jms-client package as below: If I am not wrong, I think Ivan proposed similar approach, though you give much more details above ? My only concern with this is that * It does not allow for interoperability for and via message headers. * When message headers stabilize further in hedwig, particularly for server side filtering, etc; we will need to reimplement this piece in provider. If it is just a metter of more time to thrash out the generic approach to how typed metadata will be handled in hedwig messages, we can hold off on this JIRA until that is resolved ! > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409327#comment-13409327 ] Mridul Muralidharan commented on BOOKKEEPER-310: bq. 1) protocol changes in BOOKKEEPER-309: It was my oversight that I did not realize it was a publically exposed api between modules ! Should have looked at the diff's more carefully, my apolgoes. Actually there is no reason for it to exist in protocol and can be easily moved as a private method within PublishHandler. Will that be fine ? bq. seems body is a required field for Message. is there anything special you considered? I had made body optional in an earlier change, but reverted following Ivan's comment. The commented out code slipped my cleanup. bq. I forgot to say that it would be better to have test cases to test returning message seq id for publish requests. I dont have specific testcases, but then quite a lot of activemq testcases in JMS provider module will fail if seq-id is not returned :-) Some of them even try to correlate between publisher and consumer : though I think in a clustered deployment, this will fail - so we might need to relook at these testcases in future anyway ... I will try to find a publish/subscribe() testcase in server and add an assert to check for the published seq-id. > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409321#comment-13409321 ] Mridul Muralidharan commented on BOOKKEEPER-311: I think I messed up numbering (towards the end) of the various problem in my previous comment and possibly making them difficult to comprehend ... not sure how to edit it to clean it up. In the last two lines, when I refer to Problem (2) and Problem (1), I am referring to the overall problems (not the specific ones I numbered as 1 and 2 later on). Sorry for the errors. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed as part of testing JMS provider. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-311) Changes in hedwig client api to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409319#comment-13409319 ] Mridul Muralidharan commented on BOOKKEEPER-311: bq. 1) the changes break backward compatibility. it changed the consume semantic. if existed applications used consume as async callback, it broken existed applications' assumption. one way I think we might not change consume's semantic, adding another call which returns only when consume request is written to the channel. this new call name might be 'syncConsume' or other better name. When discussing with Ivan, it looked like the existing consume() method was not consistent with rest of the api exposed from the interface all async methods were prefixed with 'async' and the sync ones were not. We assumed the implementation of consume was an oversight (particularly since it did not have a async variant) - hence the addition of an additional explicit asyncConsume. Having said that, you are right, it is an intentional interface behavioral change introduced. Note that there are two aspects to the overall change : Basically, what happens when user does * consume() and then close() (no-auto ack mode) vs * close() (ack mode - controlled via auto_send_consume_message_enabled=true (default) and consumed_messages_buffer_size (default 5) ). This in context of the fact that there is option of automatic consume acknowledgement, and buffering of these consume acknowledgements implemented in java api. (Actually, you can mix ack-mode with explicit consume() too, and vice versa btw - the latter resulting in some interesting issues). The changes to client code (on consume related changes) are to handle issues with these two cases. In the first case : If consume remains async, there are two possibilities here : 1) consume() results in request being sent to server, and so server does not redeliver the message again. 2) consume() request was in flight (within netty), while close() is executed - resulting in socket close before request makes it out. As you mentioned, we can ofcourse keep consume() as async (but with no way to track progress via a future) and introduce a syncConsume() - functionally, other than inability to track future, this will be functionally equivalent way to resolve the bug ! Note that, as you mentioned in point 2 above, other than netty 'telling us' about delivery of request to server - there is nothing much else we can do. Practically, this is good enough. It is a best case effort, and does not give transactional gaurantee's. For the second case (close() with auto-ack) : The changes to *ResponseHandler via addition of handleChannelClosedExplicitly (rename ?) is to handle this case. Ensure that buffered state (in this case, consume'd seq-id which is not yet ack'ed to server) is written before closing socket. Even this is, ofcourse, best case effort : note that this is used for sending buffered seq-id but could be used for others too (now or in future). (1) is the desired behavior, and (2) is actually fairly common - please note that this observed and then fixed, not other way around :-) I was really looking to no change hedwig in any way possible. a) The sync aspect comes in ONLY when the invocation of the method results in a request being sent to the server. b) If consume() does not result in request to server (due to buffering of consume requests), then changes to close() handle the delivery of buffered seq'id. To recap: The issue we are trying to resolve is, if message is consume'd by user, within reasonable gaurantee's, it must not be sent back to him. Problem (2) above meant that upto last 4 messages might always be sent back to client. Problem (1) meant that consume + close might send last message (or last N if batch consume by user !) back to him. > Changes in hedwig client api to support JMS spec > > > Key: BOOKKEEPER-311 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-311 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-client-consume.patch.1, > hedwig-client-publish.patch.1, hedwig-client.patch > > > Primary changes are : > a) Add support for returning seq-id for a publish request. This is an api > change (backwardly compatible for users). > b) Make consume a sync consume, with addition of an asyncConsume - this is to > ensure that invoking consume() ensure request makes to server before > returning (with what reasonable gaurantees that netty allows). > c) Ensure that explicit close'ing of session will flush buffered consume > seq-id's when auto-ack is enabled (default in hedwig java client). > In addition, there are also fixes for > d) Fix NPE's observed a
[jira] [Commented] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407756#comment-13407756 ] Mridul Muralidharan commented on BOOKKEEPER-309: .bq How is the proposal to add MessageHeader I proposed in BOOKKEEPER-78? Without type information, any attempts to filtering will be broken. More specifically, in case of JMS, there are strict rules in terms of conversion between types - which cant be inferred without the type. .bq regarding splitting into different jira's. As I explained before, and the reason I grouped the jira's as tasks under a single umbrella JIRA, is because the tasks are unfortunately interdependent. I do understand it makes reviewing them a bit of a pain, and the inability to apply them separately - but they cant be split and applied independently (except for server changes I guess) due to overlap. Unfortunately, I do not have the cycles to try to create additional 4 partial workspaces for this : which wont be functionally complete in case of (2) and (3), or untestable - in case of only (1); and unusable/broken - in case of only (4). Having said that, the individual diff's currently in the jira's loosely follow this already - (2) and (3) are mostly split along consume and publish whereever possible, but are not independently applicable due to hedwig code level overlap. Hope you understand. > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-308) Add support for JMS implementation for hedwig
[ https://issues.apache.org/jira/browse/BOOKKEEPER-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407615#comment-13407615 ] Mridul Muralidharan commented on BOOKKEEPER-308: Have attached patches to the sub-tasks, all review comments (iirc) have been incorporated. Please let me know if there is anything missing. > Add support for JMS implementation for hedwig > - > > Key: BOOKKEEPER-308 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-308 > Project: Bookkeeper > Issue Type: Improvement >Reporter: Mridul Muralidharan > > Umbrella bug to keep track of all constituent bugs related to the JMS > provider implementation for hedwig. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-309) Protocol changes in hedwig to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-309: --- Attachment: hedwig-protocol.patch.1 Protocol changes for adding metadata. It is standalone (except for hedwig-jms-client depending on it). The diffs also overlap with client publish changes - it is not possible to separate them out imo due to the generated code. > Protocol changes in hedwig to support JMS spec > -- > > Key: BOOKKEEPER-309 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-309 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-protocol.patch, hedwig-protocol.patch.1 > > > JMS spec compliance requires three changes to the protocol. > a) Support for message properties. > b) Make body optional (message contains only properties). > c) Return the published message's seq-id in the response. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-310) Changes in hedwig server to support JMS spec
[ https://issues.apache.org/jira/browse/BOOKKEEPER-310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan updated BOOKKEEPER-310: --- Attachment: hedwig-server.patch.1 Very minor changes to server compared to earlier version - essentially to revert the changes related to optional body. It is back to being mandatory now (as earlier). > Changes in hedwig server to support JMS spec > > > Key: BOOKKEEPER-310 > URL: https://issues.apache.org/jira/browse/BOOKKEEPER-310 > Project: Bookkeeper > Issue Type: Sub-task >Reporter: Mridul Muralidharan > Attachments: hedwig-server.patch, hedwig-server.patch.1 > > > The primary changes are : > a) Support modified protocol changes (optional body). > b) Return the published message's seq-id in the response. > c) Minor bugfix to Array indexing in bucket which was triggered in a testcase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
