Re: Review Request: Add 64 bit message sequence to enqueued messages
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/#review20370
---
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp
https://reviews.apache.org/r/11009/#comment41953
Just a suggestion: instead of turning sequencing on with a boolean option,
you could allow users to choose the property name they wish the sequence to be
inserted as. The default is the empty string meaning no sequencing. Gives a
little more flexibility to the user without much cost. As I say however, just a
suggestion, not a blocking issue.
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp
https://reviews.apache.org/r/11009/#comment41952
Note: Alan is reporting significant impact to performance from the
addAnnotation() method. This is just fyi really as there is as yet no
alternative.
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp
https://reviews.apache.org/r/11009/#comment41951
This means that even if you set qpid.queue_msg_sequence=False in the
arguments, sequencing will be turned on. Replacing 'true' with 'value' would I
think be better.
- Gordon Sim
On May 8, 2013, 7:30 p.m., Ernie Allen wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 8, 2013, 7:30 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Description
---
The primary purpose of this is to detect when a ring queue overwrites
messages. The client can examine the sequence number and look for gaps which
indicate that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at
1 and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber
class. Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs
-
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1478851
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1478851
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
Re: Review Request: Add 64 bit message sequence to enqueued messages
On May 9, 2013, 8:12 a.m., Gordon Sim wrote:
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp, line 743
https://reviews.apache.org/r/11009/diff/2/?file=289011#file289011line743
Note: Alan is reporting significant impact to performance from the
addAnnotation() method. This is just fyi really as there is as yet no
alternative.
$ qpid-cpp-benchmark --repeat 5 --create-option
node:{x-declare:{arguments:{'qpid.queue_msg_sequence':1}}}
send-tp recv-tp l-min l-max l-avg total-tp
30720 30671 0.3244.03 18.28 30064
31778 31729 0.2254.36 16.77 31068
31664 31637 0.3969.50 22.11 31014
31303 31301 0.3343.97 17.32 30737
31577 31403 0.3337.73 15.59 30785
$ qpid-cpp-benchmark --repeat 5
send-tp recv-tp l-min l-max l-avg total-tp
47138 47120 0.1329.02 9.3545870
46299 46156 0.1325.24 8.3344803
45910 45793 0.1353.53 10.81 45300
46607 46388 0.1629.70 9.2145245
46905 46766 0.1631.43 9.6245776
- Gordon
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/#review20370
---
On May 8, 2013, 7:30 p.m., Ernie Allen wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 8, 2013, 7:30 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Description
---
The primary purpose of this is to detect when a ring queue overwrites
messages. The client can examine the sequence number and look for gaps which
indicate that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at
1 and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber
class. Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs
-
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1478851
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1478851
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
[jira] [Created] (QPID-4823) [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue
Robbie Gemmell created QPID-4823: Summary: [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue Key: QPID-4823 URL: https://issues.apache.org/jira/browse/QPID-4823 Project: Qpid Issue Type: Bug Components: Java Broker Reporter: Robbie Gemmell Assignee: Robbie Gemmell Fix For: 0.22 add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Created] (QPID-4824) [Java AMQP 1.0] Reduce memory usage of codec
Rob Godfrey created QPID-4824: - Summary: [Java AMQP 1.0] Reduce memory usage of codec Key: QPID-4824 URL: https://issues.apache.org/jira/browse/QPID-4824 Project: Qpid Issue Type: Improvement Components: Java Common Reporter: Rob Godfrey Assignee: Rob Godfrey Priority: Minor The codec in the amqp-1-0-common code generates lots of useless hashmaps which are never used. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4823) [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue
[ https://issues.apache.org/jira/browse/QPID-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13652845#comment-13652845 ] Robbie Gemmell commented on QPID-4823: -- Change made via http://svn.apache.org/r1480581 [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue --- Key: QPID-4823 URL: https://issues.apache.org/jira/browse/QPID-4823 Project: Qpid Issue Type: Bug Components: Java Broker Reporter: Robbie Gemmell Assignee: Robbie Gemmell Fix For: 0.22 add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-4823) [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue
[ https://issues.apache.org/jira/browse/QPID-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13652846#comment-13652846 ] Robbie Gemmell commented on QPID-4823: -- r1480581 merged to the 0.22 release branch as http://svn.apache.org/r1480583 [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue --- Key: QPID-4823 URL: https://issues.apache.org/jira/browse/QPID-4823 Project: Qpid Issue Type: Bug Components: Java Broker Reporter: Robbie Gemmell Assignee: Robbie Gemmell Fix For: 0.22 add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Resolved] (QPID-4823) [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue
[ https://issues.apache.org/jira/browse/QPID-4823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robbie Gemmell resolved QPID-4823. -- Resolution: Fixed [Java Broker] add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue --- Key: QPID-4823 URL: https://issues.apache.org/jira/browse/QPID-4823 Project: Qpid Issue Type: Bug Components: Java Broker Reporter: Robbie Gemmell Assignee: Robbie Gemmell Fix For: 0.22 add missing field to web management UI to set the 'alertThresholdQueueDepthBytes' attribute when creating a queue -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Add 64 bit message sequence to enqueued messages
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 9, 2013, 12:35 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Changes
---
Changed QueueSettings initialization to use the value of
qpid.queue_msg_sequencing so declaring the queue with
qpid.queue_msg_sequence=False will not turn on sequencing
Description
---
The primary purpose of this is to detect when a ring queue overwrites messages.
The client can examine the sequence number and look for gaps which indicate
that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at 1
and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber class.
Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs (updated)
-
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1478851
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1478851
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
Re: Review Request: Add 64 bit message sequence to enqueued messages
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/#review20377
---
Like Gordon said, I have been working on adding a HA ID to messages, and there
is a dramatic performance degradation. I'm torn between optimizing
addAnnotations or handling the HA ID in a different way for now. If we both end
up needing a sequence number we should use the same one.
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp
https://reviews.apache.org/r/11009/#comment41958
Suggest renaming to qpid.seq to reduce per-message overhead.
- Alan Conway
On May 9, 2013, 12:35 p.m., Ernie Allen wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 9, 2013, 12:35 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Description
---
The primary purpose of this is to detect when a ring queue overwrites
messages. The client can examine the sequence number and look for gaps which
indicate that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at
1 and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber
class. Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs
-
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1478851
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1478851
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
[jira] [Updated] (QPID-4642) Can't complete make distcheck
[ https://issues.apache.org/jira/browse/QPID-4642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Irina Boverman updated QPID-4642: - Description: According to this: http://www.gnu.org/software/automake/manual/automake.html#Checking-the-Distribution It should be possible to use defaults as in below. Using these steps: cd qpid/cpp ./bootstrap ./configure make distcheck -- Ends up in error: -- make[1]: Entering directory `/home/jenkins/workspace/qpid-cpp-trunk-autotools-distcheck/label/master/qpid/cpp/bindings/qpid' make[1]: *** No rule to make target `perl/qpid.pm', needed by `distdir'. Stop. make[1]: Leaving directory `/home/jenkins/workspace/qpid-cpp-trunk-autotools-distcheck/label/master/qpid/cpp/bindings/qpid' make: *** [distdir] Error 1 was: According to this: http://www.gnu.org/software/automake/manual/automake.html#Checking-the-Distribution It should be possible to use defaults as in below. Using these steps: cd qpid/cpp ./bootstrap ./configure make distcheck -- Ends up in error: -- make[1]: Leaving directory `/var/lib/jenkins/workspace/qpid-cpp-trunk-distcheck/label/master/qpid/cpp/docs/api' (cd docs/man make top_distdir=../../qpidc-0.21 distdir=../../qpidc-0.21/docs/man \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[1]: Entering directory `/var/lib/jenkins/workspace/qpid-cpp-trunk-distcheck/label/master/qpid/cpp/docs/man' make[1]: *** No rule to make target `../../src/qpidd', needed by `qpidd.1'. Stop. make[1]: Leaving directory `/var/lib/jenkins/workspace/qpid-cpp-trunk-distcheck/label/master/qpid/cpp/docs/man' make: *** [distdir] Error 1 Can't complete make distcheck --- Key: QPID-4642 URL: https://issues.apache.org/jira/browse/QPID-4642 Project: Qpid Issue Type: Bug Components: C++ Broker Affects Versions: 0.21 Environment: Building on RHEL6, Server, x86_64 (automake). Reporter: Irina Boverman Assignee: Alan Conway Priority: Minor According to this: http://www.gnu.org/software/automake/manual/automake.html#Checking-the-Distribution It should be possible to use defaults as in below. Using these steps: cd qpid/cpp ./bootstrap ./configure make distcheck -- Ends up in error: -- make[1]: Entering directory `/home/jenkins/workspace/qpid-cpp-trunk-autotools-distcheck/label/master/qpid/cpp/bindings/qpid' make[1]: *** No rule to make target `perl/qpid.pm', needed by `distdir'. Stop. make[1]: Leaving directory `/home/jenkins/workspace/qpid-cpp-trunk-autotools-distcheck/label/master/qpid/cpp/bindings/qpid' make: *** [distdir] Error 1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Commented] (QPID-3838) [JMS] Vendor specific properties should be prefixed with JMS_
[ https://issues.apache.org/jira/browse/QPID-3838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13652992#comment-13652992 ] Rajith Attapattu commented on QPID-3838: I have committed the fix described in the above comment @ http://svn.apache.org/r1480656 [JMS] Vendor specific properties should be prefixed with JMS_ - Key: QPID-3838 URL: https://issues.apache.org/jira/browse/QPID-3838 Project: Qpid Issue Type: Bug Components: Java Client Affects Versions: 0.12, 0.14 Reporter: Rajith Attapattu Priority: Minor Labels: jms-compliance As per the JMS spec, vendor specific message properties should be prefixed with JMS_ Since we are including qpid.subject in all outgoing messages, it's causing a TCK failure. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Updated] (QPID-3317) JMS client doesn't distinguish between node and link bindings
[ https://issues.apache.org/jira/browse/QPID-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajith Attapattu updated QPID-3317: --- Fix Version/s: 0.20 JMS client doesn't distinguish between node and link bindings - Key: QPID-3317 URL: https://issues.apache.org/jira/browse/QPID-3317 Project: Qpid Issue Type: Bug Components: Java Client Reporter: Rajith Attapattu Assignee: Rajith Attapattu Labels: addressing Fix For: 0.20 The x-bindings specified in node properties should be created/deleted at the time the node is created/deleted. The x-bindings specified in link properties should be created/deleted at the time a link is established to a node. Ex creating producer or consumer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Add 64 bit message sequence to enqueued messages
On May 9, 2013, 1:47 p.m., Alan Conway wrote:
Like Gordon said, I have been working on adding a HA ID to messages, and
there is a dramatic performance degradation. I'm torn between optimizing
addAnnotations or handling the HA ID in a different way for now. If we both
end up needing a sequence number we should use the same one.
addAnnotation() is likely a lot better performant for amqp-1.0 than for
amqp-0_10. It would be worth testing this. It's probably not worth fixing for
0_10, it will be difficult anyway I think.
- Andrew
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/#review20377
---
On May 9, 2013, 12:35 p.m., Ernie Allen wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 9, 2013, 12:35 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Description
---
The primary purpose of this is to detect when a ring queue overwrites
messages. The client can examine the sequence number and look for gaps which
indicate that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at
1 and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber
class. Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs
-
/trunk/qpid/cpp/src/qpid/broker/Queue.h 1478851
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1478851
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1478851
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
[jira] [Commented] (QPID-4685) [Java Broker] update documentation to reflect changes to configuration etc
[ https://issues.apache.org/jira/browse/QPID-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13653014#comment-13653014 ] Alex Rudyy commented on QPID-4685: -- The changes to documentations were made in the trunk revision http://svn.apache.org/r1480672 . The changes from trunk revision r1480672 were merged into 0.22 branch under revision http://svn.apache.org/r1480684 . [Java Broker] update documentation to reflect changes to configuration etc -- Key: QPID-4685 URL: https://issues.apache.org/jira/browse/QPID-4685 Project: Qpid Issue Type: Bug Components: Documentation, Java Broker Affects Versions: 0.21 Reporter: Robbie Gemmell Assignee: Robbie Gemmell Priority: Blocker Fix For: 0.22 update documentation to reflect changes to configuration etc E.g: - Remove all trace of config.xml - Update command line option documentation to reflect additions/removal in this area. - New ACL option in QPID-4678 - Outline when it is still necessary to use an XML configuration file for a virtualhost and how to do so. Etc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
[jira] [Closed] (QPID-4685) [Java Broker] update documentation to reflect changes to configuration etc
[ https://issues.apache.org/jira/browse/QPID-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rudyy closed QPID-4685. Resolution: Fixed The required changes have been made to documentation [Java Broker] update documentation to reflect changes to configuration etc -- Key: QPID-4685 URL: https://issues.apache.org/jira/browse/QPID-4685 Project: Qpid Issue Type: Bug Components: Documentation, Java Broker Affects Versions: 0.21 Reporter: Robbie Gemmell Assignee: Robbie Gemmell Priority: Blocker Fix For: 0.22 update documentation to reflect changes to configuration etc E.g: - Remove all trace of config.xml - Update command line option documentation to reflect additions/removal in this area. - New ACL option in QPID-4678 - Outline when it is still necessary to use an XML configuration file for a virtualhost and how to do so. Etc -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
What is the correct behaviour for the following case
If we use the following address with the java client you get an error with
exchange bind.
myEx_headers;{create: always,node:{type: topic,x-declare:{type:headers}}}
However the exchange (or queue) is created, all though the bind fails.
When this happens,
1. Should we reverse the queue or exchange declare ? (in addition to
throwing an exception)
2. Leave the exchange or queue as it and just throw the exception.
We currently do #2.
What are your thoughts on this?
Regards,
Rajith
Re: What is the correct behaviour for the following case
On 05/09/2013 04:40 PM, Rajith Attapattu wrote:
If we use the following address with the java client you get an error with
exchange bind.
myEx_headers;{create: always,node:{type: topic,x-declare:{type:headers}}}
However the exchange (or queue) is created, all though the bind fails.
Is the client sending the correct bind? (Since the address doesn't
explicitly have one it should assume one where all messages would match).
When this happens,
1. Should we reverse the queue or exchange declare ? (in addition to
throwing an exception)
2. Leave the exchange or queue as it and just throw the exception.
We currently do #2.
What are your thoughts on this?
3. In this specific case I suspect the fix is to send the correct bind
so that it doesn't fail.
However in general if you have node level x-bindings and an x-declare I
don't think there is a need to make them fail 'atomically'.
-
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Re: What is the correct behaviour for the following case
On Thu, May 9, 2013 at 11:52 AM, Gordon Sim g...@redhat.com wrote:
On 05/09/2013 04:40 PM, Rajith Attapattu wrote:
If we use the following address with the java client you get an error with
exchange bind.
myEx_headers;{create: always,node:{type: topic,x-declare:{type:headers}*
*}}
However the exchange (or queue) is created, all though the bind fails.
Is the client sending the correct bind? (Since the address doesn't
explicitly have one it should assume one where all messages would match).
No the client is not sending the x-match:all like the c++ or python client.
I'm definitely going to add it.
When this happens,
1. Should we reverse the queue or exchange declare ? (in addition to
throwing an exception)
2. Leave the exchange or queue as it and just throw the exception.
We currently do #2.
What are your thoughts on this?
3. In this specific case I suspect the fix is to send the correct bind so
that it doesn't fail.
Agreed.
However in general if you have node level x-bindings and an x-declare I
don't think there is a need to make them fail 'atomically'.
Thanks, my view as well.
--**--**-
To unsubscribe, e-mail:
dev-unsubscribe@qpid.apache.**orgdev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Spec file changes to be included in 0.22? (was: License problem with qpid-python)
I've merged the changes over to the 0.22 branch. On Tue, May 7, 2013 at 5:48 PM, Justin Ross justin.r...@gmail.com wrote: This is now approved for 0.22. On Tue, May 7, 2013 at 12:52 PM, Justin Ross justin.r...@gmail.com wrote: Unless there's any technical objection, I'd like to approve this (it seems quite important to me). I'll approve QPID-4798 tonight if there are no issues. Justin On Tue, May 7, 2013 at 11:58 AM, Rafael Schloming r...@alum.mit.edu wrote: I think we should, so I'll officially request it. I've included the revisions for the necessary changes below. The bulk of these are actually producing and tweaking the stripped versions of the spec files. There was a lot of tweaks to those in order to do things like update the copyright and tweak the stripping script to produce a consistent order so we could diff the content without seeing spurious changes. As those don't touch any code and are fairly uninteresting I've included only the revisions here. They all culminate in the version of the files now present on trunk. The actual code changes are left expanded below and are all quite simple. --Rafael r1478738 r1478736 r1478582 | aconway | 2013-05-02 17:47:13 -0400 (Thu, 02 May 2013) | 5 lines QPID-4798: Fix up c++ code generation to use stripped spec files. The stripped specs have no doc elements, this was tripping up the code generator. Fixed the generator to process doc elements if they are present but to carry on without them if they are not. r1478552 r1478520 | rgodfrey | 2013-05-02 16:40:00 -0400 (Thu, 02 May 2013) | 1 line QPID-4798 : use stripped BSD licensed versions of the amqp spec files r1478449 r1478444 r1478434 r1478425 r1478400 r1478397 r1478396 r1478393 r1478385 r1478376 r1478374 r1478373 r1478093 | rhs | 2013-05-01 13:04:31 -0400 (Wed, 01 May 2013) | 1 line QPID-4798: switched python client over to using stripped XML r1478092 r1478041 r1478025 r1478012 On Tue, May 7, 2013 at 6:41 AM, Robbie Gemmell robbie.gemm...@gmail.comwrote: Hi everyone, I was wondering if we are going to include the spec file changes into 0.22? I think we should but noone seems to have made a move in that direction. Within the Java tree there is only a trivial change required, identical to trunk. I'm not sure if the same is true in the other languages. We should probably do the change at the same time if we are going to. Robbie -- Original message -- From: Rafael Schloming r...@alum.mit.edu Date: 1 May 2013 18:30 Subject: Re: License problem with qpid-python To: dev@qpid.apache.org Cc: 706...@bugs.debian.org FYI I've filed the following JIRA to track this: https://issues.apache.org/jira/browse/QPID-4798 --Rafael On Tue, Apr 30, 2013 at 10:09 AM, Thomas Goirand tho...@goirand.fr wrote: Dear QPID maintainers, Jonas Smedegaard just sent a bug report on the Debian bug tracker, because he believes that the qpid-python package in Debian is non-free: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=706101 Indeed, when having a look in the sepcs/* folder, we can see a LICENSE file which contains both the Apache-2.0 and AMQP license. Though nearly all files in that folder contains only the AMQP license header. So it is not clear at all under which license these files are. And if they are only licensed under the AMQP license, then they are non-free in the eyes of Debian (the AMQP license isn't suitable for Debian). If this issue isn't solved quickly, then the package will have to be removed from Debian. Also, since Debian Wheezy will be out this week-end, a lightning fast answer from you would be really appreciated. Best case would be if we could solve this problem before the release. Cheers, Thomas Goirand (zigo) - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: Review Request: Add 64 bit message sequence to enqueued messages
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11009/
---
(Updated May 9, 2013, 5:40 p.m.)
Review request for qpid, Alan Conway, Gordon Sim, and Justin Ross.
Changes
---
This patch is based on an updated trunk.
The annotation key under which sequence numbers are stored is now specified in
the settings.
In other words, to enable queue sequencing, declare the queue with
arguments: {'qpid.queue_msg_sequence':'user_key_name'...}
The user_key_name can be any client supplied non-empty string.
Also included are the python tests for creating the queue and retrieving the
sequence numbers.
Description
---
The primary purpose of this is to detect when a ring queue overwrites messages.
The client can examine the sequence number and look for gaps which indicate
that messages have been dropped.
To enable, create the queue with the qpid.queue_msg_sequence arg:
std::string addr(my-queue;
{create:sender, delete:always,
node: {type: queue, x-declare: {arguments:
{qpid.queue_msg_sequence:1, qpid.policy_type:ring,
qpid.max_count:4);
Sender sender = session.createSender(addr);
This enables sequencing when a message is enqueued. Sequence numbers start at 1
and are unsigned 64 bit numbers.
The exchange level sequencing uses qpid.msg_sequence as the key.
*** Question *** Should queues use the same key to enable sequencing as
exchanges? qpid.msg_sequence The page
https://cwiki.apache.org/qpid/qpid-extensions-to-amqp.html indicates that
qpid.msg_sequence is supported when declaring a queue, but that key was never
supported for queues.
To get the sequence number:
uint64_t seqNo;
seqNo = response.getProperties()[qpid.queue_msg_sequence];
*** Note ***
I added a uint64 to the queue class. There is an existing message sequence
number for queues. It is a SequenceNumber class.
However, this is a 32 bit number. I was leery of changing that to a 64 bit
number since it is used throughout the broker and not just as a msg counter.
I'm assuming a 32 bit number is not sufficient in this case since wrapping
around to zero would complicate the logic of detecting overwritten messages.
There is a public class variable in Queue.h named std::string seqNoKey; It is
unused. My guess is that it was intended for use with the SequenceNumber class.
Can it be removed?
This does not address the issue of persisting the message sequence after a
broker restart. This will be a future issue.
Before committing this, C++ and python tests would need to be added.
This addresses bug QPID-4591.
https://issues.apache.org/jira/browse/QPID-4591
Diffs (updated)
-
/trunk/qpid/cpp/src/qpid/broker/Queue.cpp 1480722
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.h 1480722
/trunk/qpid/cpp/src/qpid/broker/QueueSettings.cpp 1480722
/trunk/qpid/tests/src/py/qpid_tests/broker_0_10/new_api.py 1480722
Diff: https://reviews.apache.org/r/11009/diff/
Testing
---
See attached testring.cpp and mm (make script)
They create a ring queue that holds 4 messages with qpid.queue_msg_sequence,
sends enough messages to overflow the queue, and then retrieves the messages.
It outputs the message sequence numbers.
Thanks,
Ernie Allen
0.22 release update - RC3 is available
From revision 1480717 of the release branch: http://people.apache.org/~jross/qpid-0.22-rc3/ http://people.apache.org/~jross/qpid-0.22-rc3-testing.txt My testing on Fedora 16 x86-64 encountered no errors. This RC has all the updates for stripped spec xml files. Thanks, Rafi, Robbie, and Alan, for helping to resolve that quickly. There are still 11 bugs open against 0.21/0.22. At this point I think they need to get moved out of this release. http://s.apache.org/qpid-0.22-open-bugs I'd like to call a vote for release in about a week if no new blocking issues arise, so please provide any feedback you have sooner than later. Many thanks to those who have tested and reported their results. Justin --- 0.22 release page: https://cwiki.apache.org/qpid/022-release.html - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
Re: 0.22 release update - RC3 is available
On Thu, May 9, 2013 at 4:02 PM, Justin Ross jr...@apache.org wrote: This RC has all the updates for stripped spec xml files. Thanks, Rafi, Robbie, and Alan, for helping to resolve that quickly. Er, correction! Thanks to Rafi, *Rob*, and Alan, though I wouldn't be surprised if Robbie had a hand in it as well. - To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org For additional commands, e-mail: dev-h...@qpid.apache.org
