Re: One or Many reply queues for Clients? THANKS
Thanks very much, Paul. Steve --- Paul Clarke <[EMAIL PROTECTED]> wrote: > >Thank you for your response. I had thought about > >minimizing the number of queues by 10 times -- like > >you just suggested. But, I did not know how to go > >about it. Sorry but, I seem to fail to understand > your > >above explanation. Can you please bear with me and > >explain a bit more what you mean by the above? Hash > >value... Prime number, how are they created & used? > > >Thanks again, > > >Steve > > Steve, > > Well like I said there are many different ways you > can achieve this and I > hesitate to suggest a way to do this because this > list doesn't let one get > away with anything. So let me emphasise that this is > *one* way you *could* > do this. > > You need a constant, fairly random identifier > associated with your client. > You might have a userid + department id or whatever. > Let's say we use the > clients machine hostname. > > Now add up the numeric characters in the name. ie. > in C this could be > > char * p = HostName; /* null terminated hostname > */ > intKey = 0; > > while (*p) Key += *p++; > > > Now we have the ASCII total for the characters we > then need to convert this > into one of 10 names. Well it's a lot easier to > convert to prime numbers so > let's convert it to one of 11 names (closest prime). > > int Hash = Key % 11; > > The '%' operator gives us the modulus ie. what's > left over after division, > the remainder. > > So now we have a number, Hash, between 0 and 10 that > is fairly evenly > spread across our clients. > We can then define 11 queues, ie. REPLYQ_0, > REPLYQ_1, > REPLYQ_2..REPLYQ_10 and use the Hash number as > part of the queue name > we open. > > Does this make it any clearer ? > > Cheers, > P. > > Paul G Clarke > WebSphere MQ Development > IBM Hursley > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
>Thank you for your response. I had thought about >minimizing the number of queues by 10 times -- like >you just suggested. But, I did not know how to go >about it. Sorry but, I seem to fail to understand your >above explanation. Can you please bear with me and >explain a bit more what you mean by the above? Hash >value... Prime number, how are they created & used? >Thanks again, >Steve Steve, Well like I said there are many different ways you can achieve this and I hesitate to suggest a way to do this because this list doesn't let one get away with anything. So let me emphasise that this is *one* way you *could* do this. You need a constant, fairly random identifier associated with your client. You might have a userid + department id or whatever. Let's say we use the clients machine hostname. Now add up the numeric characters in the name. ie. in C this could be char * p = HostName; /* null terminated hostname */ intKey = 0; while (*p) Key += *p++; Now we have the ASCII total for the characters we then need to convert this into one of 10 names. Well it's a lot easier to convert to prime numbers so let's convert it to one of 11 names (closest prime). int Hash = Key % 11; The '%' operator gives us the modulus ie. what's left over after division, the remainder. So now we have a number, Hash, between 0 and 10 that is fairly evenly spread across our clients. We can then define 11 queues, ie. REPLYQ_0, REPLYQ_1, REPLYQ_2..REPLYQ_10 and use the Hash number as part of the queue name we open. Does this make it any clearer ? Cheers, P. Paul G Clarke WebSphere MQ Development IBM Hursley Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
Paul, >> It is very easy to use something relatively unique about a client to generate a hash value which can then be used to choose the reply queue to use. Even something as simple as adding up the characters in the host name and taking the modulus of an appropriate prime number would be good enough for these purposes. By changing the prime number you can then vary the storage vs contention equation. >> Thank you for your response. I had thought about minimizing the number of queues by 10 times -- like you just suggested. But, I did not know how to go about it. Sorry but, I seem to fail to understand your above explanation. Can you please bear with me and explain a bit more what you mean by the above? Hash value... Prime number, how are they created & used? Thanks again, Steve --- Paul Clarke <[EMAIL PROTECTED]> wrote: > > I am thinking that we can have either: > > 1- One reply queue that would be serviced by > all the clients > > OR > > 2- A dedicated reply queue for each client. > > > > Steve > > Steve, > > I think you're missing the obvious and perhaps > preferred alternative, that > of 'm' clients using 'n' queues. > Generally speaking it is a good idea to avoid the > extremes you mention. All > clients sharing the same queue gives you the maximum > amount of contention > on the queue which is not a good thing. Similarly, > each client having its > own queue gives you the maximum resource overhead > which is excessive since > most of the time these queues are empty. When you > only have 600 clients it > is not such a big deal but if you had 6,000 clients > then either of these > options would give you a poor result. What you > should have is say 6,000 > clients striped over, say, 60 queues. It is very > easy to use something > relatively unique about a client to generate a hash > value which can then be > used to choose the reply queue to use. Even > something as simple as adding > up the characters in the host name and taking the > modulus of an appropriate > prime number would be good enough for these > purposes. By changing the prime > number you can then vary the storage vs contention > equation. > > Cheers, > P. > > Paul G Clarke > WebSphere MQ Development > IBM Hursley > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
> I am thinking that we can have either: > 1- One reply queue that would be serviced by all the clients > OR > 2- A dedicated reply queue for each client. > > Steve Steve, I think you're missing the obvious and perhaps preferred alternative, that of 'm' clients using 'n' queues. Generally speaking it is a good idea to avoid the extremes you mention. All clients sharing the same queue gives you the maximum amount of contention on the queue which is not a good thing. Similarly, each client having its own queue gives you the maximum resource overhead which is excessive since most of the time these queues are empty. When you only have 600 clients it is not such a big deal but if you had 6,000 clients then either of these options would give you a poor result. What you should have is say 6,000 clients striped over, say, 60 queues. It is very easy to use something relatively unique about a client to generate a hash value which can then be used to choose the reply queue to use. Even something as simple as adding up the characters in the host name and taking the modulus of an appropriate prime number would be good enough for these purposes. By changing the prime number you can then vary the storage vs contention equation. Cheers, P. Paul G Clarke WebSphere MQ Development IBM Hursley Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
Jim, Never mind my question re why the applications were not deleting the prem. dyn queues before terminating. I think I know the answer: I can see that some reply queues may become orphaned, in which case they would have to be deleted outside the apps. In my design, I'll try to delete them before terminating the apps. And then run a script, like yours, periodically to delete the orphaned ones. Thanks, Steve --- Jim Ford <[EMAIL PROTECTED]> wrote: > We have a similar environment with about 600 NT > clients, and we chose > option 2. We use a single permanent dynamic model > queue as the source for > the reply queues, and the applications make the > user's ID part of the > permanent queue name. We had thought about option 1, > but it can be useful > to be able to examine the messages sent to one > specific user. It also lets > us see who's connected at any particular time by > checking IPPROCS on the > queues. > > Administration's no big deal. As I said, we only > have a single model queue. > And periodically we run a script that blindly tries > to delete all of these > reply queues. We've found that the queue manager > restarts much faster > without these 600 queues. And our application will > simply recreate the > queue the next time the user connects, so we're OK > with deleting it. If the > queue is open, or if it has messages on it, the > queue won't be deleted, so > the script just tries to delete everything. > > > > > Steve Muller > <[EMAIL PROTECTED]>To: > [EMAIL PROTECTED] > Sent by: MQSeriescc: > List > Subject: One or Many reply queues for Clients? > N.AC.AT> > > > 07/05/2002 10:17 > AM > Please respond to > MQSeries List > > > > > > > Hi all, > > Part of our MQ environment is OS/390 (2.1) and NT > (5.2). We will have about 190 NT clients connected > to > OS/390. These NT client apps will send a request and > get their replies from OS/390, based on a correlid. > More than 90% of these requests will be queries. We > are estimating that, for all the 190 clients > combined, > there will be about 50 requests and roughly > 50-1 replies per sec. The turn-around time is > expected by the user to be 2-12 sec based on the > importance of the request. > > I am thinking that we can have either: > 1- One reply queue that would be serviced by > all the > clients, OR > 2- A dedicated reply queue for each client. > > In the scenario 1: I know that a qualified GET > would be slower from a queue with a high Q-depth (I > know indexing could be used ). As far as I can see, > the major downside to this approach would be if > something goes wrong with this queue all the clients > would suffer. > > In the scenario 2: If we have a dedicated reply > queue > for each client, MQ admin would not be very happy > but > the retrieval of the msgs would be faster (even if > we > used qualified Gets). And a problem with one queue > would not effect the rest of the reply queues > > > What do you think? Your insights would be much > appreciated. > > Regards, > > Steve > > > > > __ > Do You Yahoo!? > Sign up for SBC Yahoo! Dial - First Month Free > http://sbc.yahoo.com > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
Jim, and others, thanks for your responses... Jim: I think we'll be using Perm Dyn. queues as you suggested. I was just wondering about the reasoning behind why your applications are not deleting their respective prem. dyn. queues before they terminate-- instead of running a script periodically for that same purpose. Can you please get back to me on that? Thanks, SM --- Jim Ford <[EMAIL PROTECTED]> wrote: > We have a similar environment with about 600 NT > clients, and we chose > option 2. We use a single permanent dynamic model > queue as the source for > the reply queues, and the applications make the > user's ID part of the > permanent queue name. We had thought about option 1, > but it can be useful > to be able to examine the messages sent to one > specific user. It also lets > us see who's connected at any particular time by > checking IPPROCS on the > queues. > > Administration's no big deal. As I said, we only > have a single model queue. > And periodically we run a script that blindly tries > to delete all of these > reply queues. We've found that the queue manager > restarts much faster > without these 600 queues. And our application will > simply recreate the > queue the next time the user connects, so we're OK > with deleting it. If the > queue is open, or if it has messages on it, the > queue won't be deleted, so > the script just tries to delete everything. > > > > > Steve Muller > <[EMAIL PROTECTED]>To: > [EMAIL PROTECTED] > Sent by: MQSeriescc: > List > Subject: One or Many reply queues for Clients? > N.AC.AT> > > > 07/05/2002 10:17 > AM > Please respond to > MQSeries List > > > > > > > Hi all, > > Part of our MQ environment is OS/390 (2.1) and NT > (5.2). We will have about 190 NT clients connected > to > OS/390. These NT client apps will send a request and > get their replies from OS/390, based on a correlid. > More than 90% of these requests will be queries. We > are estimating that, for all the 190 clients > combined, > there will be about 50 requests and roughly > 50-1 replies per sec. The turn-around time is > expected by the user to be 2-12 sec based on the > importance of the request. > > I am thinking that we can have either: > 1- One reply queue that would be serviced by > all the > clients, OR > 2- A dedicated reply queue for each client. > > In the scenario 1: I know that a qualified GET > would be slower from a queue with a high Q-depth (I > know indexing could be used ). As far as I can see, > the major downside to this approach would be if > something goes wrong with this queue all the clients > would suffer. > > In the scenario 2: If we have a dedicated reply > queue > for each client, MQ admin would not be very happy > but > the retrieval of the msgs would be faster (even if > we > used qualified Gets). And a problem with one queue > would not effect the rest of the reply queues > > > What do you think? Your insights would be much > appreciated. > > Regards, > > Steve > > > > > __ > Do You Yahoo!? > Sign up for SBC Yahoo! Dial - First Month Free > http://sbc.yahoo.com > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive > > Instructions for managing your mailing list > subscription are provided in > the Listserv General Users Guide available at > http://www.lsoft.com > Archive: http://vm.akh-wien.ac.at/MQSeries.archive __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
Hello Steve: I feel this is quite difficult design problem for the MQSeries. I am not sure what MQSeries Pub/Sub is (we use another MOM for publish/subscribe problems) but you might want to check if it supports any kind of multicast -- if it does, consider using it. Otherwise, multiple queues sound much better to me, especially if the publisher has enough information to set the proper correlation ID for the client: In this case, you might want to create queues on demand from publishers using PCF messages and have one "signalling" queue for clients to learn about the names of their respective queues. In this case, the administrator will not have much to be unhappy about (except for the security, of course, but that would be another question... :-)). Have a nice day, Pavel Steve Muller <[EMAIL PROTECTED]>To: [EMAIL PROTECTED] Sent by: MQSeriescc: List Subject: One or Many reply queues for Clients? 07/05/2002 11:17 AM Please respond to MQSeries List Hi all, Part of our MQ environment is OS/390 (2.1) and NT (5.2). We will have about 190 NT clients connected to OS/390. These NT client apps will send a request and get their replies from OS/390, based on a correlid. More than 90% of these requests will be queries. We are estimating that, for all the 190 clients combined, there will be about 50 requests and roughly 50-1 replies per sec. The turn-around time is expected by the user to be 2-12 sec based on the importance of the request. I am thinking that we can have either: 1- One reply queue that would be serviced by all the clients, OR 2- A dedicated reply queue for each client. In the scenario 1: I know that a qualified GET would be slower from a queue with a high Q-depth (I know indexing could be used ). As far as I can see, the major downside to this approach would be if something goes wrong with this queue all the clients would suffer. In the scenario 2: If we have a dedicated reply queue for each client, MQ admin would not be very happy but the retrieval of the msgs would be faster (even if we used qualified Gets). And a problem with one queue would not effect the rest of the reply queues What do you think? Your insights would be much appreciated. Regards, Steve __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive -- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
We have a similar environment with about 600 NT clients, and we chose option 2. We use a single permanent dynamic model queue as the source for the reply queues, and the applications make the user's ID part of the permanent queue name. We had thought about option 1, but it can be useful to be able to examine the messages sent to one specific user. It also lets us see who's connected at any particular time by checking IPPROCS on the queues. Administration's no big deal. As I said, we only have a single model queue. And periodically we run a script that blindly tries to delete all of these reply queues. We've found that the queue manager restarts much faster without these 600 queues. And our application will simply recreate the queue the next time the user connects, so we're OK with deleting it. If the queue is open, or if it has messages on it, the queue won't be deleted, so the script just tries to delete everything. Steve Muller <[EMAIL PROTECTED]>To: [EMAIL PROTECTED] Sent by: MQSeriescc: List Subject: One or Many reply queues for Clients? 07/05/2002 10:17 AM Please respond to MQSeries List Hi all, Part of our MQ environment is OS/390 (2.1) and NT (5.2). We will have about 190 NT clients connected to OS/390. These NT client apps will send a request and get their replies from OS/390, based on a correlid. More than 90% of these requests will be queries. We are estimating that, for all the 190 clients combined, there will be about 50 requests and roughly 50-1 replies per sec. The turn-around time is expected by the user to be 2-12 sec based on the importance of the request. I am thinking that we can have either: 1- One reply queue that would be serviced by all the clients, OR 2- A dedicated reply queue for each client. In the scenario 1: I know that a qualified GET would be slower from a queue with a high Q-depth (I know indexing could be used ). As far as I can see, the major downside to this approach would be if something goes wrong with this queue all the clients would suffer. In the scenario 2: If we have a dedicated reply queue for each client, MQ admin would not be very happy but the retrieval of the msgs would be faster (even if we used qualified Gets). And a problem with one queue would not effect the rest of the reply queues What do you think? Your insights would be much appreciated. Regards, Steve __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: One or Many reply queues for Clients?
Not an answer but more an observation. Rather than the 190 queues you may want to consider grouping - say 19 queues, each serviced by 10 clients. Neil -Original Message- From: Steve Muller [mailto:[EMAIL PROTECTED]] Sent: Fri 05/07/2002 11:17 To: [EMAIL PROTECTED] Cc: Subject: One or Many reply queues for Clients? Hi all, Part of our MQ environment is OS/390 (2.1) and NT (5.2). We will have about 190 NT clients connected to OS/390. These NT client apps will send a request and get their replies from OS/390, based on a correlid. More than 90% of these requests will be queries. We are estimating that, for all the 190 clients combined, there will be about 50 requests and roughly 50-1 replies per sec. The turn-around time is expected by the user to be 2-12 sec based on the importance of the request. I am thinking that we can have either: 1- One reply queue that would be serviced by all the clients, OR 2- A dedicated reply queue for each client. In the scenario 1: I know that a qualified GET would be slower from a queue with a high Q-depth (I know indexing could be used ). As far as I can see, the major downside to this approach would be if something goes wrong with this queue all the clients would suffer. In the scenario 2: If we have a dedicated reply queue for each client, MQ admin would not be very happy but the retrieval of the msgs would be faster (even if we used qualified Gets). And a problem with one queue would not effect the rest of the reply queues What do you think? Your insights would be much appreciated. Regards, Steve __ Do You Yahoo!? Sign up for SBC Yahoo! Dial - First Month Free http://sbc.yahoo.com Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
