[jira] [Assigned] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey reassigned QPID-4171: --- Assignee: Philip Harvey (was: Keith Wall) persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Philip Harvey Attachments: QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Deadlock
Hello, We are having a DeadLock issue and it seems that this is the same bug as here: https://issues.apache.org/jira/browse/QPID-3216 So here what the logs say: [2012-08-01 23:24:00.355] INFO sample_service.SampleService org.apache.qpid.client.BasicMessageConsumer Closing consumer:7[2130838978] [2012-08-01 23:24:00.355] ERROR Thread-71System.err Exception in thread Thread-71 java.lang.RuntimeException: java.lang.InterruptedException [2012-08-01 23:24:00.355] ERROR Thread-71System.err at org.apache.qpid.client.AMQSession.syncDispatchQueue(AMQSession.java:2215) [2012-08-01 23:24:00.355] ERROR Thread-71System.err at org.apache.qpid.client.BasicMessageConsumer_0_10.getMessageFromQueue(BasicMessageConsumer_0_10.java:436) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at org.apache.qpid.client.BasicMessageConsumer.receive(BasicMessageConsumer.java:407) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at com.moodys.msp.lb.ServiceRequestPoller.run(ServiceRequestPoller.java:43) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.lang.Thread.run(Thread.java:680) [2012-08-01 23:24:00.356] ERROR Thread-71System.err Caused by: java.lang.InterruptedException [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:979) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at org.apache.qpid.client.AMQSession.syncDispatchQueue(AMQSession.java:2211) [2012-08-01 23:24:00.356] ERROR Thread-71System.err ... 4 more Also here is the JConsole output for the deadlock: Name: Dispatcher-Channel-0 State: BLOCKED on java.lang.Object@c58f769 owned by: main Total blocked: 2 Total waited: 1 Stack trace: org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:571) org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:535) com.eugene.package.CallResponseListener.tryCloseMessageConsumer(CallResponseListener.java:55) com.eugene.package.CallResponseListener.onMessage(CallResponseListener.java:50) org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:167) org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:699) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:205) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:47) org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3388) org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3332) - locked java.lang.Object@22fe135d - locked java.lang.Object@1a0283e org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3113) org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3106) org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:55) org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3260) java.lang.Thread.run(Thread.java:680) And on the other side: Name: main State: BLOCKED on java.lang.Object@22fe135d owned by: Dispatcher-Channel-0 Total blocked: 148 Total waited: 133 Stack trace: org.apache.qpid.client.AMQSession.close(AMQSession.java:731) - locked java.lang.Object@c58f769 org.apache.qpid.client.AMQSession.close(AMQSession.java:708) org.apache.qpid.client.AMQSession.close(AMQSession.java:567) come.eugene.package.MSPConnectionImpl.close(MSPConnectionImpl.java:54) com.eugene.package.ToDelete.doRoundRobinTest(ToDelete.java:62) com.eugene.package.ToDelete.main(ToDelete.java:25) Any luck of that bug being closed?
[jira] [Updated] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey updated QPID-4171: Status: Ready To Review (was: In Progress) persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Philip Harvey Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey updated QPID-4171: Attachment: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch attached patch persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Philip Harvey Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Assigned] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey reassigned QPID-4171: --- Assignee: Keith Wall (was: Philip Harvey) please commit the patch I attached today persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Keith Wall Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426520#comment-13426520 ] Alex Rudyy commented on QPID-4171: -- I reviewed the patch and the changes look fine for me. persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Keith Wall Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4170) JMX threads can spin in the Queue MBean if there is a failure when viewing message content
[ https://issues.apache.org/jira/browse/QPID-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426542#comment-13426542 ] Justin Ross commented on QPID-4170: --- Reviewed by Keith. Approved for 0.18. JMX threads can spin in the Queue MBean if there is a failure when viewing message content -- Key: QPID-4170 URL: https://issues.apache.org/jira/browse/QPID-4170 Project: Qpid Issue Type: Bug Components: Java Broker Affects Versions: 0.12, 0.13, 0.14, 0.15, 0.16, 0.17, 0.18 Reporter: Robbie Gemmell Assignee: Keith Wall Priority: Critical Fix For: 0.19 During the work on QPID-4164 it was found that JMX threads can spin in the Queue MBean if they fail when viewing message content, looping down into the message and back store asking for content which will never be returned. These threads will never stop spinning, and will consume CPU resources and introduce additional/unnecessary resource contention within the store layer. The view content method of the Queue MBean should be updated to avoid this possibility. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4167) Queue attributes are ignored on queue creation with web management console
[ https://issues.apache.org/jira/browse/QPID-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426544#comment-13426544 ] Justin Ross commented on QPID-4167: --- Reviewed by Keith and Robbie. Approved for 0.18. Queue attributes are ignored on queue creation with web management console -- Key: QPID-4167 URL: https://issues.apache.org/jira/browse/QPID-4167 Project: Qpid Issue Type: Bug Components: Java Broker Affects Versions: 0.18, 0.19 Reporter: Alex Rudyy Assignee: Robbie Gemmell Fix For: 0.19 Attachments: 0001-QPID-4167-Fix-setting-of-queue-attributes-via-web-ma.patch Setting of such attributes like queueFlowControlSizeBytes, queueFlowResumeSizeBytes, alertThresholdMessageAge, alertThresholdMessageSize, alertThresholdQueueDepthMessages, alertRepeatGap on queue creation with web management console does not work. These attributes are ignored on queue creation with web management console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Deadlock
Hi Eugene, What is the version of the client you are using? Is it 0.10 client? Kind Regards, Alex On 1 August 2012 11:47, eugene eugen.ra...@gmail.com wrote: Hello, We are having a DeadLock issue and it seems that this is the same bug as here: https://issues.apache.org/jira/browse/QPID-3216 So here what the logs say: [2012-08-01 23:24:00.355] INFO sample_service.SampleService org.apache.qpid.client.BasicMessageConsumer Closing consumer:7[2130838978] [2012-08-01 23:24:00.355] ERROR Thread-71System.err Exception in thread Thread-71 java.lang.RuntimeException: java.lang.InterruptedException [2012-08-01 23:24:00.355] ERROR Thread-71System.err at org.apache.qpid.client.AMQSession.syncDispatchQueue(AMQSession.java:2215) [2012-08-01 23:24:00.355] ERROR Thread-71System.err at org.apache.qpid.client.BasicMessageConsumer_0_10.getMessageFromQueue(BasicMessageConsumer_0_10.java:436) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at org.apache.qpid.client.BasicMessageConsumer.receive(BasicMessageConsumer.java:407) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at com.moodys.msp.lb.ServiceRequestPoller.run(ServiceRequestPoller.java:43) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.lang.Thread.run(Thread.java:680) [2012-08-01 23:24:00.356] ERROR Thread-71System.err Caused by: java.lang.InterruptedException [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:979) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:207) [2012-08-01 23:24:00.356] ERROR Thread-71System.err at org.apache.qpid.client.AMQSession.syncDispatchQueue(AMQSession.java:2211) [2012-08-01 23:24:00.356] ERROR Thread-71System.err ... 4 more Also here is the JConsole output for the deadlock: Name: Dispatcher-Channel-0 State: BLOCKED on java.lang.Object@c58f769 owned by: main Total blocked: 2 Total waited: 1 Stack trace: org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:571) org.apache.qpid.client.BasicMessageConsumer.close(BasicMessageConsumer.java:535) com.eugene.package.CallResponseListener.tryCloseMessageConsumer(CallResponseListener.java:55) com.eugene.package.CallResponseListener.onMessage(CallResponseListener.java:50) org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:725) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:167) org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:699) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:205) org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:47) org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3388) org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3332) - locked java.lang.Object@22fe135d - locked java.lang.Object@1a0283e org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3113) org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3106) org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:55) org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3260) java.lang.Thread.run(Thread.java:680) And on the other side: Name: main State: BLOCKED on java.lang.Object@22fe135d owned by: Dispatcher-Channel-0 Total blocked: 148 Total waited: 133 Stack trace: org.apache.qpid.client.AMQSession.close(AMQSession.java:731) - locked java.lang.Object@c58f769 org.apache.qpid.client.AMQSession.close(AMQSession.java:708) org.apache.qpid.client.AMQSession.close(AMQSession.java:567) come.eugene.package.MSPConnectionImpl.close(MSPConnectionImpl.java:54) com.eugene.package.ToDelete.doRoundRobinTest(ToDelete.java:62) com.eugene.package.ToDelete.main(ToDelete.java:25) Any luck of that bug being closed? Thank You, Eugene. -- View this message in context: http://apache-qpid-developers.2158895.n2.nabble.com/Deadlock-tp7581518.html Sent from the Apache Qpid developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org -
Re: Deadlock
Yup, its 0.10 -- View this message in context: http://apache-qpid-developers.2158895.n2.nabble.com/Deadlock-tp7581518p7581526.html Sent from the Apache Qpid developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4152) Refactor coverage targets in Java build system
[ https://issues.apache.org/jira/browse/QPID-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey updated QPID-4152: Attachment: 0001-QPID-4152-added-condition-to-cover-test-target-so-it.patch Attached patch to fix problem running ant cover-test. Added condition to cover-test target so it only runs if units tests exist. Was previously choking trying to run amqp-1-0-common tests, because junit.jar wasn't on classpath. Refactor coverage targets in Java build system -- Key: QPID-4152 URL: https://issues.apache.org/jira/browse/QPID-4152 Project: Qpid Issue Type: Improvement Components: Ant Build System Reporter: Keith Wall Assignee: Philip Harvey Priority: Minor Labels: cobertura Fix For: 0.19 Attachments: 0001-QPID-4152-added-condition-to-cover-test-target-so-it.patch 1) Eliminate a hard-coded list used to determine those modules for which coverage is generated. 2) Eliminate duplication between the test target and the cover-test targets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4152) Refactor coverage targets in Java build system
[ https://issues.apache.org/jira/browse/QPID-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Harvey updated QPID-4152: Assignee: Keith Wall (was: Philip Harvey) Status: Open (was: Ready To Review) further change is required (see patch). Pls commit it if you're happy with it. Refactor coverage targets in Java build system -- Key: QPID-4152 URL: https://issues.apache.org/jira/browse/QPID-4152 Project: Qpid Issue Type: Improvement Components: Ant Build System Reporter: Keith Wall Assignee: Keith Wall Priority: Minor Labels: cobertura Fix For: 0.19 Attachments: 0001-QPID-4152-added-condition-to-cover-test-target-so-it.patch 1) Eliminate a hard-coded list used to determine those modules for which coverage is generated. 2) Eliminate duplication between the test target and the cover-test targets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: qpidd refactor
On July 31, 2012, 9:33 p.m., Alan Conway wrote: /trunk/qpid/cpp/src/qpid/ha/ReplicatingSubscription.cpp, line 268 https://reviews.apache.org/r/5833/diff/5/?file=131497#file131497line268 Comment typo: qm.position should be m.getSequeunce() Thanks! Fixed. On July 31, 2012, 9:33 p.m., Alan Conway wrote: /trunk/qpid/cpp/src/qpid/ha/QueueGuard.cpp, line 71 https://reviews.apache.org/r/5833/diff/5/?file=131493#file131493line71 The completion isn't really part of the persistence context. Is there a more generic message context they could be part of? I agree its slightly misplaced. It is where it is because the store requires access to it through the PersistableMessage interface and I was keen to avoid the changes rippling too far. However, that doesn't mean that's the only interface through which it can be obtained. I have massaged this a little to (I hope) fix the concern. - Gordon --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5833/#review9667 --- On July 31, 2012, 8:18 p.m., Gordon Sim wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5833/ --- (Updated July 31, 2012, 8:18 p.m.) Review request for qpid, Alan Conway and Kenneth Giusti. Description --- == Background == I've been looking at what would be required to get AMQP 1.0 support in the qpidd broker (using proton-c). In that context I felt there was a need to refactor the broker code, particularly that part that would be shared between different protocol versions. Part of the motivation was clearer separation of 0-10 specific logic, so that 1.0 logic could be introduced as an alternative. However part of it was also simply the recognition of some long-standing problems that we have never stopped to address. So, here is a patch representing my ideas on what is needed. This is a little stale again (patch was generated against r13613342) and needs (yet) another rebase. However it is getting to the point where I'll be asking to commit it soon, so if anyone has feedback, now is the time to give it! == Key Changes == qpid::broker::Message This is now supposed to be a protocol neutral representation of a message. It no longer exposes qpid::framing::FrameSet. It can be based on data received in different encodings (this patch only includes the existing 0-10 encoding). The immutable, sharable state is separated from the mutable queue-specific state. Messages themselves are no longer held through a shared pointer but are passed by reference or copied if needed. The immutable state (essentially the data as received) *is* still shared and referenced internally through an intrusive pointer. There is no longer a message level lock. A message instance is 'owned' by someother entity (usually the queue it is on) which controls concurrent access/modification if necessary. The persistence context is a separate part of the message also. Currently that can be shared between two message instances if desired. qpid::broker::Messages Switched from using qpid::broker::QueuedMessage (which relied on shared pointer to message itself and made sequence number the explicit - and only - way to refer to a specific message) to using modified Message class directly and a new qpid::broker::QueueCursor. The cursor is opaque outside the Messages implementation to which it relates. It provides a way to refer to a specific message (without directly using sequence number, though at present that is what is used 'under the covers') and/or to track progress through a sequence of messages (for consumers or other iterating entities). I.e. its an iterator to a Message within its containing Messages instance that is not invalidated by changes to that container. A Messages instance *owns* the Message instances within it. Other classes access this through a reference or (raw) pointer, or if needed copy it (the immutable part can be - and is - safely shared). The codepath for browse/consume is a lot more unified now. You use a cursor and call Messages::next() in each case. This also lays the foundation for selectors. The simplified Messages interface led to a simplied MessageDistributor. There is still a little more to do to clarify these separate roles (or indeed perhaps unify them?) but more on that later. qpid::broker::amqp_0_10::MessageTransfer This represents the familiar 0-10 encoding of a message. This class is broadly similar to the old Message class, based on a FrameSet. However it represents the shared and essentially immutable state. The sendHeader() method now explicitly takes a copy of the original headers and adds to
Re: Review Request: qpidd refactor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5833/ --- (Updated Aug. 1, 2012, 12:38 p.m.) Review request for qpid, Alan Conway and Kenneth Giusti. Changes --- Minor adjustments per Alan's feedback. Description --- == Background == I've been looking at what would be required to get AMQP 1.0 support in the qpidd broker (using proton-c). In that context I felt there was a need to refactor the broker code, particularly that part that would be shared between different protocol versions. Part of the motivation was clearer separation of 0-10 specific logic, so that 1.0 logic could be introduced as an alternative. However part of it was also simply the recognition of some long-standing problems that we have never stopped to address. So, here is a patch representing my ideas on what is needed. This is a little stale again (patch was generated against r13613342) and needs (yet) another rebase. However it is getting to the point where I'll be asking to commit it soon, so if anyone has feedback, now is the time to give it! == Key Changes == qpid::broker::Message This is now supposed to be a protocol neutral representation of a message. It no longer exposes qpid::framing::FrameSet. It can be based on data received in different encodings (this patch only includes the existing 0-10 encoding). The immutable, sharable state is separated from the mutable queue-specific state. Messages themselves are no longer held through a shared pointer but are passed by reference or copied if needed. The immutable state (essentially the data as received) *is* still shared and referenced internally through an intrusive pointer. There is no longer a message level lock. A message instance is 'owned' by someother entity (usually the queue it is on) which controls concurrent access/modification if necessary. The persistence context is a separate part of the message also. Currently that can be shared between two message instances if desired. qpid::broker::Messages Switched from using qpid::broker::QueuedMessage (which relied on shared pointer to message itself and made sequence number the explicit - and only - way to refer to a specific message) to using modified Message class directly and a new qpid::broker::QueueCursor. The cursor is opaque outside the Messages implementation to which it relates. It provides a way to refer to a specific message (without directly using sequence number, though at present that is what is used 'under the covers') and/or to track progress through a sequence of messages (for consumers or other iterating entities). I.e. its an iterator to a Message within its containing Messages instance that is not invalidated by changes to that container. A Messages instance *owns* the Message instances within it. Other classes access this through a reference or (raw) pointer, or if needed copy it (the immutable part can be - and is - safely shared). The codepath for browse/consume is a lot more unified now. You use a cursor and call Messages::next() in each case. This also lays the foundation for selectors. The simplified Messages interface led to a simplied MessageDistributor. There is still a little more to do to clarify these separate roles (or indeed perhaps unify them?) but more on that later. qpid::broker::amqp_0_10::MessageTransfer This represents the familiar 0-10 encoding of a message. This class is broadly similar to the old Message class, based on a FrameSet. However it represents the shared and essentially immutable state. The sendHeader() method now explicitly takes a copy of the original headers and adds to it or otherwise modifies it if needed (e.g. for redelivered flag, ttl, annotations etc). [Ideally I'd like to move more of the 0-10 specific classes out of qpid::broker and into qpid::broker::amqp_0_10, but that has no functional relevance so I've left existing classes alone for now.] qpid::broker::Consumer The deliver() method now takes a QueueCursor (representing a 'handle' to this message for use in subsequent operations such as accept, relese etc) and a *constant reference* to the Message itself (i.e. consumers can't alter the state of the message on the queue directly, but only through operations on the queue itself). qpid::broker::QueueRegistry The actual queue creation has been pulled out into a base class, QueueFactory. The actual class of the Queue returned can now be varied and there are two subclasses in the current patch. The first is a replacement for the ring policy logic, whereby messages are removed from the queue in order to keep the queue from growing above a configured limit. The second is for last value queues and simply pulls the special case behaviour out of the common code path. The handling of queue configuration has also been made cleaner and more uniform, based on the QueueSettings class.
[jira] [Updated] (QPID-4170) JMX threads can spin in the Queue MBean if there is a failure when viewing message content
[ https://issues.apache.org/jira/browse/QPID-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated QPID-4170: - Affects Version/s: (was: 0.18) Fix Version/s: (was: 0.19) 0.18 Assignee: Robbie Gemmell (was: Keith Wall) Change merged to the 0.18 branch. JMX threads can spin in the Queue MBean if there is a failure when viewing message content -- Key: QPID-4170 URL: https://issues.apache.org/jira/browse/QPID-4170 Project: Qpid Issue Type: Bug Components: Java Broker Affects Versions: 0.12, 0.13, 0.14, 0.15, 0.16, 0.17 Reporter: Robbie Gemmell Assignee: Robbie Gemmell Priority: Critical Fix For: 0.18 During the work on QPID-4164 it was found that JMX threads can spin in the Queue MBean if they fail when viewing message content, looping down into the message and back store asking for content which will never be returned. These threads will never stop spinning, and will consume CPU resources and introduce additional/unnecessary resource contention within the store layer. The view content method of the Queue MBean should be updated to avoid this possibility. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-4167) Queue attributes are ignored on queue creation with web management console
[ https://issues.apache.org/jira/browse/QPID-4167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell updated QPID-4167: - Affects Version/s: (was: 0.19) (was: 0.18) 0.17 Fix Version/s: (was: 0.19) 0.18 Changes now merged to the 0.18 branch. Queue attributes are ignored on queue creation with web management console -- Key: QPID-4167 URL: https://issues.apache.org/jira/browse/QPID-4167 Project: Qpid Issue Type: Bug Components: Java Broker Affects Versions: 0.17 Reporter: Alex Rudyy Assignee: Robbie Gemmell Fix For: 0.18 Attachments: 0001-QPID-4167-Fix-setting-of-queue-attributes-via-web-ma.patch Setting of such attributes like queueFlowControlSizeBytes, queueFlowResumeSizeBytes, alertThresholdMessageAge, alertThresholdMessageSize, alertThresholdQueueDepthMessages, alertRepeatGap on queue creation with web management console does not work. These attributes are ignored on queue creation with web management console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426615#comment-13426615 ] Keith Wall commented on QPID-4171: -- Patch applied. persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Keith Wall Fix For: 0.19 Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (QPID-4171) persistent messages from a non-transactional session can be enqueued out of order
[ https://issues.apache.org/jira/browse/QPID-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall resolved QPID-4171. -- Resolution: Fixed Fix Version/s: 0.19 persistent messages from a non-transactional session can be enqueued out of order - Key: QPID-4171 URL: https://issues.apache.org/jira/browse/QPID-4171 Project: Qpid Issue Type: Bug Components: Java Broker, Java Broker BDB Store Affects Versions: 0.16, 0.18, 0.19 Reporter: Keith Wall Assignee: Keith Wall Fix For: 0.19 Attachments: 0001-QPID-4171-Fix-enqueue-ordering-for-persistent-messsa.patch, QPID-4171-scribbles.txt A defect in AsyncAutoCommitTransaction means that occasionally messages delivered by a non-transactional JMS are seen to be enqueued out-of-order and therefore delivered to a consumer out of the expected order. It is this problem that underlies QPID-4014, although as I think this problem is wider than ConflationQueues, I am raising this separately. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Closed] (QPID-4014) System test ConflationQueueTest.testParallelProductionAndConsumption failed
[ https://issues.apache.org/jira/browse/QPID-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall closed QPID-4014. Resolution: Fixed System test ConflationQueueTest.testParallelProductionAndConsumption failed --- Key: QPID-4014 URL: https://issues.apache.org/jira/browse/QPID-4014 Project: Qpid Issue Type: Bug Components: Java Tests Reporter: Philip Harvey Assignee: Philip Harvey Priority: Minor Fix For: 0.17 Attachments: 0001-QPID-4014-increase-debug-logging-in-ConflationQueueL.patch, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.2nd-occurence.gz, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.build291.gz, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.build419.bz2, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.gz Unit test ConflationQueueTest.testParallelProductionAndConsumption has failed on Jenkins at least twice: * [Build 206|https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-BDB-TestMatrix/jdk=JDK%201.6%20(latest),label=Ubuntu,profile=java-bdb.0-9-1/206/testReport/org.apache.qpid.server.queue/ConflationQueueTest/testParallelProductionAndConsumption/] * [Build 291|https://builds.apache.org/job/Qpid-Java-Java-BDB-TestMatrix/291/jdk=JDK%201.6%20(latest),label=Ubuntu,profile=java-bdb.0-9-1/testReport/junit/org.apache.qpid.server.queue/ConflationQueueTest/testParallelProductionAndConsumption/] This test has previously run successfully. On first glance, the test doesn't appear to have any inherent race conditions, therefore it is possible that this represents a genuine problem. In this test, two producers each each 400 messages with alternating conflation key values, followed by special shutdown messages. A single consumer remembers the latest message for each conflation key value. Each producer's last few messages should be: * Message 397 (conflation key '1') * Message 398 (conflation key '0') * Message 399 (conflation key '1') * Shutdown message The assertion in question checks the last message received for each conflation key value. In this case, the last message with key '1' is message 397, not the expected message 399. Examination of the test output (attached) indicates that the consumer received message 399 once (it must have been conflated), and that - surprisingly - it arrived *after* the two shutdown messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4152) Refactor coverage targets in Java build system
[ https://issues.apache.org/jira/browse/QPID-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426619#comment-13426619 ] Keith Wall commented on QPID-4152: -- Reviewed, no comments. Patch applied. Refactor coverage targets in Java build system -- Key: QPID-4152 URL: https://issues.apache.org/jira/browse/QPID-4152 Project: Qpid Issue Type: Improvement Components: Ant Build System Reporter: Keith Wall Assignee: Keith Wall Priority: Minor Labels: cobertura Fix For: 0.19 Attachments: 0001-QPID-4152-added-condition-to-cover-test-target-so-it.patch 1) Eliminate a hard-coded list used to determine those modules for which coverage is generated. 2) Eliminate duplication between the test target and the cover-test targets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4014) System test ConflationQueueTest.testParallelProductionAndConsumption failed
[ https://issues.apache.org/jira/browse/QPID-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13426617#comment-13426617 ] Keith Wall commented on QPID-4014: -- Fixed by QPID-4171. System test ConflationQueueTest.testParallelProductionAndConsumption failed --- Key: QPID-4014 URL: https://issues.apache.org/jira/browse/QPID-4014 Project: Qpid Issue Type: Bug Components: Java Tests Reporter: Philip Harvey Assignee: Philip Harvey Priority: Minor Fix For: 0.17 Attachments: 0001-QPID-4014-increase-debug-logging-in-ConflationQueueL.patch, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.2nd-occurence.gz, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.build291.gz, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.build419.bz2, TEST-org.apache.qpid.server.queue.ConflationQueueTest.testParallelProductionAndConsumption.out.gz Unit test ConflationQueueTest.testParallelProductionAndConsumption has failed on Jenkins at least twice: * [Build 206|https://builds.apache.org/view/M-R/view/Qpid/job/Qpid-Java-Java-BDB-TestMatrix/jdk=JDK%201.6%20(latest),label=Ubuntu,profile=java-bdb.0-9-1/206/testReport/org.apache.qpid.server.queue/ConflationQueueTest/testParallelProductionAndConsumption/] * [Build 291|https://builds.apache.org/job/Qpid-Java-Java-BDB-TestMatrix/291/jdk=JDK%201.6%20(latest),label=Ubuntu,profile=java-bdb.0-9-1/testReport/junit/org.apache.qpid.server.queue/ConflationQueueTest/testParallelProductionAndConsumption/] This test has previously run successfully. On first glance, the test doesn't appear to have any inherent race conditions, therefore it is possible that this represents a genuine problem. In this test, two producers each each 400 messages with alternating conflation key values, followed by special shutdown messages. A single consumer remembers the latest message for each conflation key value. Each producer's last few messages should be: * Message 397 (conflation key '1') * Message 398 (conflation key '0') * Message 399 (conflation key '1') * Shutdown message The assertion in question checks the last message received for each conflation key value. In this case, the last message with key '1' is message 397, not the expected message 399. Examination of the test output (attached) indicates that the consumer received message 399 once (it must have been conflated), and that - surprisingly - it arrived *after* the two shutdown messages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (QPID-4152) Refactor coverage targets in Java build system
[ https://issues.apache.org/jira/browse/QPID-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keith Wall resolved QPID-4152. -- Resolution: Fixed Refactor coverage targets in Java build system -- Key: QPID-4152 URL: https://issues.apache.org/jira/browse/QPID-4152 Project: Qpid Issue Type: Improvement Components: Ant Build System Reporter: Keith Wall Assignee: Keith Wall Priority: Minor Labels: cobertura Fix For: 0.19 Attachments: 0001-QPID-4152-added-condition-to-cover-test-target-so-it.patch 1) Eliminate a hard-coded list used to determine those modules for which coverage is generated. 2) Eliminate duplication between the test target and the cover-test targets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Please include QPID-4171 in Qpid 0.18
Hi Justin, I would like to request the inclusion of this commit in 0.18: http://svn.apache.org/viewvc?view=revisionrevision=1367990 The commit is for this JIRA: https://issues.apache.org/jira/browse/QPID-4171. This addresses a serious Java Broker bug whereby messages delivered by a non-transactional JMS session are sometimes enqueued out of order and therefore delivered to a consumer out of the expected order. The commit also includes a new unit test for this bug, and the removal of some dead code. The changes were made by Alex, Robbie and myself, and reviewed by Keith. Thanks, Phil This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email.
Re: Deadlock
Hi Eugen, I am sorry for the late response. There is a good chance that the issue was fixed in QPID-3911 for 0.16 qpid client. However, I am afraid that you might run into other deadlocks by closing the consumer from MessageListener. I would avoid doing so. Do you have any test reproducing the issue? Kind Regards, Alex On 1 August 2012 13:10, eugene eugen.ra...@gmail.com wrote: Yup, its 0.10 -- View this message in context: http://apache-qpid-developers.2158895.n2.nabble.com/Deadlock-tp7581518p7581526.html Sent from the Apache Qpid developers mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Change IO buffer ownership strategy to avoid memory leaks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6257/#review9702 --- /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp https://reviews.apache.org/r/6257/#comment20671 Should this be using a max-frame-size from somewhere? /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp https://reviews.apache.org/r/6257/#comment20672 What guarantees this assertion will pass? Looks like AsynchIO::getQueuedBuffer() can return 0. How does the new code grow the number of buffers if required? - Alan Conway On July 31, 2012, 10:15 p.m., Andrew Stitcher wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6257/ --- (Updated July 31, 2012, 10:15 p.m.) Review request for qpid, Alan Conway, Gordon Sim, and Steve Huston. Description --- This change reworks the buffer handling of the IO layer of qpid. Essentially it makes the AsyncIO class the owner of the IO buffers always, previously it only owned the buffers when they were on one of its queues. This allowed the buffers to leak if they were not returned to its queues for some reason (perhaps an exception being thrown). In order to apply this change to the SSL code as well I've had to bring SslConnector in line with the current TCPConnector code which it has markedly diverged from. I'm particularly seeking review on the Windows code which has been lightly tested, and the SSL code because the change is large there (although it brings the SSL code much more in line with the TCP code). This addresses bug QPID-4180. https://issues.apache.org/jira/browse/QPID-4180 Diffs - /trunk/qpid/cpp/src/qpid/client/SslConnector.cpp 1367696 /trunk/qpid/cpp/src/qpid/client/TCPConnector.h 1367696 /trunk/qpid/cpp/src/qpid/client/TCPConnector.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/AsynchIO.h 1367696 /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslHandler.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslIo.h 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslIo.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/AsynchIO.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/SslAsynchIO.h 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/SslAsynchIO.cpp 1367696 Diff: https://reviews.apache.org/r/6257/diff/ Testing --- Builds on Fedora 16, and Windows make check cmake test Both pass Except ha_tests which fail in the same way on the equivalent trunk version for me. Linux performace using qpid-cpp-benchmark seems comparable. Thanks, Andrew Stitcher
Re: Allocating ports in tests
On Tue, 2012-07-31 at 13:12 -0400, Justin Ross wrote: I recommend reserving a port, chosen at random, in the dynamic port range: * 0 - 1023 Well Known Ports * 1024 - 49151 Registered Ports * 49152 - 65535 Dynamic and/or Private Ports (from http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html) How do you avoid clashing with non-qpid processes using the dynamic range? Then, make it persistent: store the port with an appropriate key under /var/tmp (or your os equivalent), and look for it again at the start of subsequent test runs. How do you protect that file from concurrent access? I tend to think using --port 0 is less good than having the port selection logic *outside* the daemon. That way you don't have to do any hard work to discover what the daemon actually chose. If that is the only factor to choose between two approaches then I absolutely agree. For the majority of our tests however (those that don't attempt to restart a broker on the same port) I don't see an out-of-broker solution that is nearly as simple or reliable as bind(0). For the tests that do restart brokers on the same port the clashes are pretty rare. I've only seen them in overnight test where 1000s of broker restarts give 1 or 2 clashes. Still not 0 though :( Justin On Tue, 31 Jul 2012, Alan Conway wrote: I'm looking for ideas about how to allocate ports for tests. Our test suite generally runs brokers with --port 0 in order to pick an available port. That works well in most cases. The problem case I have is the new HA code. Brokers and clients of a cluster expect the cluster to have fixed addresses ports. If I kill and restart a broker during a test it needs to come up on the same port. Currently the HA tests use --port 0 and hope for the best, which works almost all the time. It does however give some spurious test failures, because some other process grabs the port in the time between stopping qpidd and restarting it. Does anyone know a good, simple scheme to allocate ports like this? It needs to allow multiple test runs on the same host without collisions. Cheers, Alan. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: qpidd refactor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/5833/ --- (Updated Aug. 1, 2012, 7:47 p.m.) Review request for qpid, Alan Conway and Kenneth Giusti. Changes --- Minor store related fixes in Queue.cpp exposed by durable federation tests. Description --- == Background == I've been looking at what would be required to get AMQP 1.0 support in the qpidd broker (using proton-c). In that context I felt there was a need to refactor the broker code, particularly that part that would be shared between different protocol versions. Part of the motivation was clearer separation of 0-10 specific logic, so that 1.0 logic could be introduced as an alternative. However part of it was also simply the recognition of some long-standing problems that we have never stopped to address. So, here is a patch representing my ideas on what is needed. This is a little stale again (patch was generated against r13613342) and needs (yet) another rebase. However it is getting to the point where I'll be asking to commit it soon, so if anyone has feedback, now is the time to give it! == Key Changes == qpid::broker::Message This is now supposed to be a protocol neutral representation of a message. It no longer exposes qpid::framing::FrameSet. It can be based on data received in different encodings (this patch only includes the existing 0-10 encoding). The immutable, sharable state is separated from the mutable queue-specific state. Messages themselves are no longer held through a shared pointer but are passed by reference or copied if needed. The immutable state (essentially the data as received) *is* still shared and referenced internally through an intrusive pointer. There is no longer a message level lock. A message instance is 'owned' by someother entity (usually the queue it is on) which controls concurrent access/modification if necessary. The persistence context is a separate part of the message also. Currently that can be shared between two message instances if desired. qpid::broker::Messages Switched from using qpid::broker::QueuedMessage (which relied on shared pointer to message itself and made sequence number the explicit - and only - way to refer to a specific message) to using modified Message class directly and a new qpid::broker::QueueCursor. The cursor is opaque outside the Messages implementation to which it relates. It provides a way to refer to a specific message (without directly using sequence number, though at present that is what is used 'under the covers') and/or to track progress through a sequence of messages (for consumers or other iterating entities). I.e. its an iterator to a Message within its containing Messages instance that is not invalidated by changes to that container. A Messages instance *owns* the Message instances within it. Other classes access this through a reference or (raw) pointer, or if needed copy it (the immutable part can be - and is - safely shared). The codepath for browse/consume is a lot more unified now. You use a cursor and call Messages::next() in each case. This also lays the foundation for selectors. The simplified Messages interface led to a simplied MessageDistributor. There is still a little more to do to clarify these separate roles (or indeed perhaps unify them?) but more on that later. qpid::broker::amqp_0_10::MessageTransfer This represents the familiar 0-10 encoding of a message. This class is broadly similar to the old Message class, based on a FrameSet. However it represents the shared and essentially immutable state. The sendHeader() method now explicitly takes a copy of the original headers and adds to it or otherwise modifies it if needed (e.g. for redelivered flag, ttl, annotations etc). [Ideally I'd like to move more of the 0-10 specific classes out of qpid::broker and into qpid::broker::amqp_0_10, but that has no functional relevance so I've left existing classes alone for now.] qpid::broker::Consumer The deliver() method now takes a QueueCursor (representing a 'handle' to this message for use in subsequent operations such as accept, relese etc) and a *constant reference* to the Message itself (i.e. consumers can't alter the state of the message on the queue directly, but only through operations on the queue itself). qpid::broker::QueueRegistry The actual queue creation has been pulled out into a base class, QueueFactory. The actual class of the Queue returned can now be varied and there are two subclasses in the current patch. The first is a replacement for the ring policy logic, whereby messages are removed from the queue in order to keep the queue from growing above a configured limit. The second is for last value queues and simply pulls the special case behaviour out of the common code path. The handling of queue configuration has also been made cleaner and more
Re: Review Request: Change IO buffer ownership strategy to avoid memory leaks
On Aug. 1, 2012, 6:40 p.m., Alan Conway wrote: /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp, line 90 https://reviews.apache.org/r/6257/diff/1/?file=131555#file131555line90 Should this be using a max-frame-size from somewhere? I agree it shouldn't be a magic number (I just copied the old magic number!). This is on the listen side and the buffers have to be set up before doing any negotiation to find out the max frame size, so for simplicity we use the maximum possible frame size. On the connect side we do in fact use the max-frame-size option to size the buffers. It would be possible though tricky to use some full sized buffers until we know the max-frame-size and then allocate all the buffers of this size for the connection. On Aug. 1, 2012, 6:40 p.m., Alan Conway wrote: /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp, line 99 https://reviews.apache.org/r/6257/diff/1/?file=131555#file131555line99 What guarantees this assertion will pass? Looks like AsynchIO::getQueuedBuffer() can return 0. How does the new code grow the number of buffers if required? The guarantee is that this connection has not ever sent anything on this connection before and could only have received a single frame - this code path is only used when sending out the initial protocol header. The new code has no mechanism for adding new buffers. This is probably an advantage as it does not allow memory growth due to outgoing message frames. If you can't get a buffer to write a frame you have to wait until one is available. In practice the write side pretty much only uses one buffer currently as AsynchIO::writable() calls up into AsynchHandler::idle() which picks up a buffer encodes into it then queues it for writing and returns. Now AsynchIO::writable() writes this buffer and calls up again until nothing is queued after the upcall. [The AsynchIO code allows for queuing of buffers to write, but it's never actually used in the current code paths. Actually I think in the current code paths you might only need a single buffer for read and a single buffer for write] - Andrew --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6257/#review9702 --- On July 31, 2012, 10:15 p.m., Andrew Stitcher wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6257/ --- (Updated July 31, 2012, 10:15 p.m.) Review request for qpid, Alan Conway, Gordon Sim, and Steve Huston. Description --- This change reworks the buffer handling of the IO layer of qpid. Essentially it makes the AsyncIO class the owner of the IO buffers always, previously it only owned the buffers when they were on one of its queues. This allowed the buffers to leak if they were not returned to its queues for some reason (perhaps an exception being thrown). In order to apply this change to the SSL code as well I've had to bring SslConnector in line with the current TCPConnector code which it has markedly diverged from. I'm particularly seeking review on the Windows code which has been lightly tested, and the SSL code because the change is large there (although it brings the SSL code much more in line with the TCP code). This addresses bug QPID-4180. https://issues.apache.org/jira/browse/QPID-4180 Diffs - /trunk/qpid/cpp/src/qpid/client/SslConnector.cpp 1367696 /trunk/qpid/cpp/src/qpid/client/TCPConnector.h 1367696 /trunk/qpid/cpp/src/qpid/client/TCPConnector.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/AsynchIO.h 1367696 /trunk/qpid/cpp/src/qpid/sys/AsynchIOHandler.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/posix/AsynchIO.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslHandler.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslIo.h 1367696 /trunk/qpid/cpp/src/qpid/sys/ssl/SslIo.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/AsynchIO.cpp 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/SslAsynchIO.h 1367696 /trunk/qpid/cpp/src/qpid/sys/windows/SslAsynchIO.cpp 1367696 Diff: https://reviews.apache.org/r/6257/diff/ Testing --- Builds on Fedora 16, and Windows make check cmake test Both pass Except ha_tests which fail in the same way on the equivalent trunk version for me. Linux performace using qpid-cpp-benchmark seems comparable. Thanks, Andrew Stitcher
[jira] [Created] (QPID-4181) proton porting work on Windows
Cliff Jansen created QPID-4181: -- Summary: proton porting work on Windows Key: QPID-4181 URL: https://issues.apache.org/jira/browse/QPID-4181 Project: Qpid Issue Type: Improvement Components: Proton Affects Versions: Future Environment: Windows Reporter: Cliff Jansen Assignee: Cliff Jansen Fix For: 0.19 Basic porting work of proton for C99 compliant compilers on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Review Request: patches for mingw
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6302/ --- Review request for qpid, Andrew Stitcher, Kenneth Giusti, Steve Huston, and Rafael Schloming. Description --- This patch set works with a recent mingw32, cmake 2.8.1, python 2.5, swig 2.0.7. A push-button build is still a ways off. The custom_commands in the cmake script to generate the protocol headers don't work yet on Windows. The most intrusive change was the introduction of a pn_socket_t type to hold a socket on both Windows and Posix platforms. An attempt was made to minimize the use of #ifdefs and split platform code into separate posix and windows directories, as is done for the C++ code. There is so little needed at the moment, this may be overkill. The qpid-proton-posix library was ditched and combined with the main qpid-proton library. Instead, the work is done in CMake to assemble the correct shared and platform specific sources as is done in the C++ tree. The driver.c implementation is proof of concept using Winsock select(). Future work would most likely replace this with an I/O completion port implementation. 1. mkdir ...\trunk\proton-c\build 2. set env vars as per trunk\config.sh set PATH=C:\Program Files (x86)\CMake 2.8\bin;C:\python25;C:\mingw_ptn\bin;C:\Windows\System32;c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build set PYTHONPATH=c:\cj\work\amqp\proton\mingw4\trunk\tests;c:\cj\work\amqp\proton\mingw4\trunk\proton-c;c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build\bindings\python set PYTHON_BINDINGS=c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build\bindings\python set PROTON_HOME=C:\cj\work\amqp\proton\mingw4\trunk 3. generate the headers: cd trunk\proton-c\build python c:\cj\work\amqp\proton\mingw4\trunk\proton-c\src\codec\encodings.h.py encodings.h python c:\cj\work\amqp\proton\mingw4\trunk\proton-c\src\protocol.h.py protocol.h 4. build cmake -G MinGW Makefiles -DSWIG_EXECUTABLE=C:\swigwin-2.0.7\swig.exe c:\cj\work\amqp\proton\mingw4\trunk\proton-c mingw32-make python ..\..\tests\proton-test This addresses bug QPID-4181. https://issues.apache.org/jira/browse/QPID-4181 Diffs - http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/CMakeLists.txt 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/bindings/CMakeLists.txt 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/bindings/python/python.i 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/include/proton/driver.h 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/include/proton/types.h 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/codec/codec.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/dispatcher/dispatcher.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/framing/framing.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/message/message.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/messenger.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/proton-dump.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/network.h PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/posix/time.c PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/uuid.h PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/windows/driver.c PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/windows/time.c PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/windows/uuid.c PRE-CREATION http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/util.c 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/tests/proton_tests/messenger.py 1368240 Diff: https://reviews.apache.org/r/6302/diff/ Testing --- Fedora, Windows Thanks, Cliff Jansen
Re: Review Request: patches for mingw
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6302/#review9728 --- I'm pretty happy with the way this is shaping up just a few nits below. http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/CMakeLists.txt https://reviews.apache.org/r/6302/#comment20710 Perhaps it makes sense to move src/driver.c into src/sys/posix/driver.c? http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/include/proton/types.h https://reviews.apache.org/r/6302/#comment20712 I think it might be better to keep pn_socket_t out of a header file that will be widely included by arbitrary client code if it is going to include a windows system header file (with their name space polluting ways and general large size). Perhaps pn_socket_t is defined with the driver API? http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/sys/uuid.h https://reviews.apache.org/r/6302/#comment20715 It seems likely to me that you don't need to #include windows.h in this header file - just including it in the actual implementation file would stop it contaminating everything with windows symbols. http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/src/util.c https://reviews.apache.org/r/6302/#comment20713 Note that strings.h is a purely Unix thing not in the C standard library. So the there should be a test like #ifdef __unix__ around it, something like: #ifdef __unix__ # include strings.h #elif defined(_WIN32) # define strcasecmp(a,b) _stricmp((a), (b)) #else # error #endif Although it might be simply better to avoid these not very standard calls and use tolower() and strcmp(). In fact since I can see you've done that work lower down these lines can probably be removed completely. - Andrew Stitcher On Aug. 1, 2012, 9:23 p.m., Cliff Jansen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/6302/ --- (Updated Aug. 1, 2012, 9:23 p.m.) Review request for qpid, Andrew Stitcher, Kenneth Giusti, Steve Huston, and Rafael Schloming. Description --- This patch set works with a recent mingw32, cmake 2.8.1, python 2.5, swig 2.0.7. A push-button build is still a ways off. The custom_commands in the cmake script to generate the protocol headers don't work yet on Windows. The most intrusive change was the introduction of a pn_socket_t type to hold a socket on both Windows and Posix platforms. An attempt was made to minimize the use of #ifdefs and split platform code into separate posix and windows directories, as is done for the C++ code. There is so little needed at the moment, this may be overkill. The qpid-proton-posix library was ditched and combined with the main qpid-proton library. Instead, the work is done in CMake to assemble the correct shared and platform specific sources as is done in the C++ tree. The driver.c implementation is proof of concept using Winsock select(). Future work would most likely replace this with an I/O completion port implementation. 1. mkdir ...\trunk\proton-c\build 2. set env vars as per trunk\config.sh set PATH=C:\Program Files (x86)\CMake 2.8\bin;C:\python25;C:\mingw_ptn\bin;C:\Windows\System32;c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build set PYTHONPATH=c:\cj\work\amqp\proton\mingw4\trunk\tests;c:\cj\work\amqp\proton\mingw4\trunk\proton-c;c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build\bindings\python set PYTHON_BINDINGS=c:\cj\work\amqp\proton\mingw4\trunk\proton-c\build\bindings\python set PROTON_HOME=C:\cj\work\amqp\proton\mingw4\trunk 3. generate the headers: cd trunk\proton-c\build python c:\cj\work\amqp\proton\mingw4\trunk\proton-c\src\codec\encodings.h.py encodings.h python c:\cj\work\amqp\proton\mingw4\trunk\proton-c\src\protocol.h.py protocol.h 4. build cmake -G MinGW Makefiles -DSWIG_EXECUTABLE=C:\swigwin-2.0.7\swig.exe c:\cj\work\amqp\proton\mingw4\trunk\proton-c mingw32-make python ..\..\tests\proton-test This addresses bug QPID-4181. https://issues.apache.org/jira/browse/QPID-4181 Diffs - http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/CMakeLists.txt 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/bindings/CMakeLists.txt 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/bindings/python/python.i 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/include/proton/driver.h 1368240 http://svn.apache.org/repos/asf/qpid/proton/trunk/proton-c/include/proton/types.h 1368240
[jira] [Commented] (QPID-3677) Asserting does not work correctly from JMS
[
https://issues.apache.org/jira/browse/QPID-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13427074#comment-13427074
]
Michael Chrostowski commented on QPID-3677:
---
Due to the exception Rob posted above I cannot consume on any topic defined by
an address in Java. I'm still trying to find a work around that doesn't involve
editing Qpid code, maybe bindURL based constructor will let it work.
Basically queueName is not being set in the Destination and the
createConsumer() call requires it, thus the NPE.
Asserting does not work correctly from JMS
--
Key: QPID-3677
URL: https://issues.apache.org/jira/browse/QPID-3677
Project: Qpid
Issue Type: Bug
Components: Java Client
Affects Versions: 0.14
Reporter: Gordon Sim
Labels: addressing
When there is an exchange named abc, the following address should result in
an error being raised:
'abc; {assert: always, node: {type: queue}}'
There is indeed an error raised, but the message is incorrect:
org.apache.qpid.AMQException: The name 'abc' supplied in the address
doesn't resolve to an exchange or a queue
More importantly however, changing the node type to topic should result in
the resolution and checking passing without error. This is not the case
however. Using:
'abc; {assert: always, node: {type: topic}}'
we get exactly the same error.
Attempting to assert on the exchange type (assuming abc is of type topic):
'abc; {assert: always, node: {type: topic, x-declare:{type:fanout}}}'
should result in an error due to mismatched expectations on exchange type.
However we get the same incorrect exception as above, i.e:
org.apache.qpid.AMQException: The name 'abc' supplied in the address
doesn't resolve to an exchange or a queue
And again, if we change the address so that it should pass:
'abc; {assert: always, node: {type: topic, x-declare:{type:topic}}}'
we then get:
Exception in thread main java.lang.NullPointerException
at
org.apache.qpid.client.AMQSession_0_10.sendConsume(AMQSession_0_10.java:626)
at
org.apache.qpid.client.AMQSession_0_10.sendConsume(AMQSession_0_10.java:83)
at
org.apache.qpid.client.AMQSession.consumeFromQueue(AMQSession.java:2592)
at
org.apache.qpid.client.AMQSession.registerConsumer(AMQSession.java:2928)
at org.apache.qpid.client.AMQSession.access$500(AMQSession.java:120)
at org.apache.qpid.client.AMQSession$4.execute(AMQSession.java:2048)
at org.apache.qpid.client.AMQSession$4.execute(AMQSession.java:2006)
at
org.apache.qpid.client.AMQConnectionDelegate_0_10.executeRetrySupport(AMQConnectionDelegate_0_10.java:346)
at
org.apache.qpid.client.AMQConnection.executeRetrySupport(AMQConnection.java:574)
at
org.apache.qpid.client.failover.FailoverRetrySupport.execute(FailoverRetrySupport.java:102)
at
org.apache.qpid.client.AMQSession.createConsumerImpl(AMQSession.java:2004)
at org.apache.qpid.client.AMQSession.createConsumer(AMQSession.java:979)
at org.apache.qpid.example.Drain.init(Drain.java:70)
at org.apache.qpid.example.Drain.main(Drain.java:103)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
