Gordon Sim wrote:
On 03/31/2010 05:41 PM, Alan Conway wrote:
I think we can extend qpid_send and qpid_recv to do most of the things
that perftest and latencytest do. Interested in any ideas on the topic.
I like the approach.
+1
Here's what I propose:
Additional qpid_send options:
--report=Y/N: report sender throughput on exit.
--report-interval=SECS: print throughput report every N seconds.
FYI qpid-bench does this based on # of messages as opposed to time (not
sure which is preferred or if both make sense in some scenarios).
--timestamp=Y/N: timestamp each message.
--rate=MSGS/SEC: send at this rate, 0 means fast as possible.
--padding=BYTES: add this much padding to messages.
--time-limit=SECS: stop after this time, 0 means no limit.
--max-noreply=N: don't send more than N messages without receiving a
reply.
Creates a reply queue and sets reply_to on every N/2 messages.
In addition you'd need to configure things like sender/receiver
capacity, sync/async, etc.
qpid_send currently reads in message content from std::in. This can be
very useful for ad-hoc tests or scripting different types of test.
However for performance you would most likely want to avoid any extra
overhead and have the message content generated by the tool itself.
Additional qpid_recv options:
--report=Y/N: report latency and throughput statistics on exit.
Latency is only calculated for messages with a timestamp.
--report-interval=SECS: print latency and throughput statistics every N
seconds.
New qpid_recv behavior: if reply-to is set, echo the message to that
address.
I could see two different types of useful behaviour here. One as above,
where you echo the message. Another would simply be to send an empty
message correlated by correlation-id. E.g. for the case where the
original message is very large and you only wish the response to
indicate that it was received (could have received message sie in the
response message in some way). Of course keeping it simple to begin with
is also fine!
If we want the flexibility I would suggest putting some sort of opcode
in the original message that would instruct the receiver how to respond,
e.g. echo the original message, send an empty (but correlated) message,
or even send a message with the receiver's current stats in it.
qpid_recv also prints out each message to std::out and again this would
need to be optional to avoid impacting perf numbers. The python drain
example has a nice feature where you can specify what aspect of each
message gets printed. Adding that to qpid_recv would be valuable.
The --max-noreply=N option is a simple form of end-to-end throughput.
The sender sets reply-to on every N/2 messages. It won't send more than
N messages beyond the last message that was replied to. The receiver
replies whenever reply-to is set.
Multiple instances of sender & receiver can be mixed and matched however
you want, to measure e.g. shared queue, pub-sub etc. scenarios. I'm
deliberately avoiding the do-everything swiss army knife approach of
perftest, that results in an overly complex test that is not as flexible
as the separate sender/receiver approach.
One thing that would be useful is to have multiple sessions per
connection for send/recv. This would be useful even for performance
testing, though it does complicate the reporting a little.
Overall I'm not sure we should try to cram too much into qpid-send and
qpid-recv. I think it might make sense to have two sets of utilities
with the difference being based on whether the utilities themselves
generate the messages, or whether they're simply used to send/recv
messages generated outside the utilities. I think the use cases around
these two scenarios are fairly different.
The things that generate/consume their own content are more oriented
towards diagnostics/testing/benchmarking, and I would expect to have
options around frequency, size, type of content, etc, whereas the things
that send/recv existing content would have more options around reading
in messages from various formats, writing them out, etc.
I propose thinking in terms of qpid-send/qpid-recv which are basically
command line versions of the API used to send/recv message content
generated outside the utility (that content could come by file,
command-line args, stdin, etc), and then qpid-source/qpid-sink which are
used as artificial sources/sinks for messages and more oriented towards
testing/diagnostics/performance.
--Rafael
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscr...@qpid.apache.org
