Thanks to both of you for looking at this. I read up on MQOO_BIND_ON_OPEN, and it appears that it will let me keep the messages together as long as they are messages on the same queue. Correct?
Our design, though, has two correlated messages, each on a different queue. The reason we have two messages is because one is string format, and one is binary. The reason we used two queues instead of one is that it's simpler programming. Plus this is a high volume application, and the messages do on occasion pile up on the queues. Since the target queue manager runs on Solaris, there's no indexing available and we were worried that if we had all of this on a single queue we'd have performance problems as depth increased.
I was hoping that our first stab at clustering wouldn't require noteworthy application changes, nor a workload exit. I suppose that we could just change the sending application to use a single queue with MQOO_BIND_ON_OPEN, and then have a simple "router" application on the back end that split the messages to the pair of local queues. That would let the back end application work without change.
But these are very big persistent messages; the binary message can be as large as 20 meg. I hate to do all that unnecessary work and unecessary logging, so I'm not optimistic that we want to pursue clustering for this, I guess.
| Neil Casey <[EMAIL PROTECTED]>
Sent by: MQSeries List <[EMAIL PROTECTED]> 10/04/2004 06:18 PM
|
|
Actually Jim, you can set up Tim's solution so that the messages are always
routed together. If your application specifies MQOO_BIND_ON_OPEN, then the
route selection is made when the queue is opened, not when the message is
PUT. This means that your application should open the queue, put the 2 (or
more) messages, and then commit and close the queue. All the messages will
be sent to the same instance. Next time you open the queue (in a new
transaction) the IBM code or your workload balancing exit can choose a new
destination instance.
When considering whether clustering to multiple servers is worthwhile, you
need to think about more than just reliability. Although the unix systems
are reliable, there are times when you want to shut them down (hardware or
OS maintenance, application software upgrades, site power outages etc etc).
Clustering also allows you to incrementally add capability to your server
cluster. When you need to add processing capacity, you don't have to
upgrade you E10K to an E15K, with all the associated extra costs. You can
just go out an buy a new V880 and add that on, so you get horizontal
scalability for your app, which is generally a very good thing. (Note, I
have used Sun systems as an example, but of course it applies to Intel,
Sun, RS6000 or whatever equally).
Regards,
Neil Casey
National Australia Bank
Southern Star Technology
WebSphere MQ Support
1/122 Lewis Rd Wantirna South
office. +61 3 9886 2375 (x82375)
mobile. +61 414 615 334
Jim Ford
<[EMAIL PROTECTED]
M> To
Sent by: MQSeries [EMAIL PROTECTED]
List cc
<[EMAIL PROTECTED]
n.AC.AT> Subject
Re: Clustering w/ Message Affinity
05/10/2004 08:55
Please respond to
MQSeries List
<[EMAIL PROTECTED]
n.AC.AT>
But whether we put to one or two queues, we have the same issue, right? The
messages may be delivered to different queue managers if we cluster.
I'm not optimistic that this problem can be solved in a way to make
clustering worthwhile. What we gain by having redundant queue managers we
give up in complexity. And I'm thinking that Unix and the hardware it runs
on are more reliable than our applications.
Tim Armstrong
<[EMAIL PROTECTED]>
Sent by: MQSeries List To
<[EMAIL PROTECTED]> [EMAIL PROTECTED]
.ac.at
cc
10/04/2004 05:35 PM
Subject
Re: Clustering w/
Please respond to Message Affinity
MQSeries List <[EMAIL PROTECTED]>
Maybe rewrite the App so it puts both messages to the same queue as
follows.
Putting App
Msg1 MsgId=None(so new MsgId gets generated) Correlid=0x0102030405 or some
other unique value <- dont use a character string as MsgId's and CorrelId's
do not get translated which can cause platform dependcy issues.
Msg2 MsgId=None CorrelId=Msg1.MsgId
Getting App
Get Msg1 by using MsgId=None(to select any MsgId) and CorrelId=0x0102030405
Get Msg2 by using MsgId=None and CorrelId=Msg1.MsgId
Kind of a generic solution to different type of messages on the same queue.
Also permits Msg3,4,5 etc. with CorrelId=Msg1.MsgID where Msg1 is a control
message and the number of other messages is unknown till run time.
Let us know which route you choose. I'm afraid there's likely to be pain
which ever way you go. Even the above can have problems in that the queue
will need to be indexed to get reasonable performance if there are a large
number of messages on the queue at any one time, which of course has an
overhead of its own :(
Regards
Tim A
-----Original Message-----
From: Jim Ford [mailto:[EMAIL PROTECTED]
Sent: Tuesday, 5 October 2004 8:09 AM
To: [EMAIL PROTECTED]
Subject: Re: Clustering w/ Message Affinity
I guess that'll work, but it's quite a bit messier than we were hoping for.
"Potkay, Peter M (ISD, IT)"
<[EMAIL PROTECTED]>
Sent by: MQSeries List
<[EMAIL PROTECTED]> To
[EMAIL PROTECTED]
en.ac.at
10/04/2004 04:34 PM cc
Subject
Please respond to Re: Clustering
MQSeries List <[EMAIL PROTECTED]> w/ Message
Affinity
dang, you are right. Forgot about that.
Well, your app will need to maintain a list of available QMs, pick one
randomly, and specify the name on the MQOD for the MQOPEN for both queues.
You could maintain an MQ Namelist of all these QMs, so the app wouldn't
have to change as you add / remove QMs. At least you will get the benefit
of reduced Administration for queues and channels.
-----Original Message-----
From: Jim Ford [mailto:[EMAIL PROTECTED]
Sent: Monday, October 04, 2004 5:23 PM
To: [EMAIL PROTECTED]
Subject: Re: Clustering w/ Message Affinity
Can you get a COA if you haven't commited? It seems like there isn't an A
to C until after you commit.
"Potkay, Peter M (ISD, IT)"
<[EMAIL PROTECTED]>
Sent by: MQSeries List
<[EMAIL PROTECTED]> To
[EMAIL PROTECTED]
en.ac.at
10/04/2004 04:01 PM cc
Subject
Please respond to Re: Clustering
MQSeries List <[EMAIL PROTECTED]> w/ Message
Affinity
You could MQOPEN Q1, put message1 with a COA, read the COA to see what QM
got Message1, and then MQOPEN Q2 specifying the QM where the COA came from,
put message2 then commit both and close both queues.
Now both messages will always be on Q1 / Q2 on the same QM. But you will
get a random QM each time.
-----Original Message-----
From: Jim Ford [mailto:[EMAIL PROTECTED]
Sent: Monday, October 04, 2004 4:55 PM
To: [EMAIL PROTECTED]
Subject: Clustering w/ Message Affinity
We don't use clustering at our site, but are considering it for a couple of
high volume applications. Some of these applications, though, use
correlation ID to match messages on two different queues.
To be more specific, a binary message is put to Q1 under syncpoint. A
second message is put to Q2. The second message's correlation ID is set to
the first message's message ID. Then a commit is done. The getting app gets
the message from Q1, and gets the correlated message from Q2 and processes
them. This works fine.
But this design requires that both messages have to be delivered to the
same target queue manager. How can we guarantee that if we use clustering?
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com Archive:
http://vm.akh-wien.ac.at/MQSeries.archive
This communication, including attachments, is for the exclusive use of
addressee and may contain proprietary, confidential or privileged
information. If you are not the intended recipient, any use, copying,
disclosure, dissemination or distribution is strictly prohibited. If
you are not the intended recipient, please notify the sender
immediately by return email and delete this communication and destroy all
copies.
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com Archive:
http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com Archive:
http://vm.akh-wien.ac.at/MQSeries.archive
This email and any attachments may contain privileged and confidential
information and are intended for the named addressee only. If you have
received this e-mail in error, please notify the sender and delete this
e-mail immediately. Any confidentiality, privilege or copyright is not
waived or lost because this e-mail has been sent to you in error. It is
your responsibility to check this e-mail and any attachments for viruses.
No warranty is made that this material is free from computer virus or any
other defect or error. Any loss/damage incurred by using this material is
not the sender's responsibility. The sender's entire liability will be
limited to resupplying the material.
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com Archive:
http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
