Neil, I am specifically thinking of the scenario where the message is already on the destination QM. Regardless of whether the messages were in syncpoint or not as they traveled across the channel, I am only concerned about how messages are handled in a HUB queue manager once they are committed to the HUB queue manager. On a busy HUB, at any given moment, I will have thousands of messages being made available every minute to the QM.
At the risk of hijacking this thread into a discussion on channel speed: I chose a speed of normal on my channels because for the longest time, we had a batch interval of zero and a batch size of 50 for all our channels. Then one of our channels started losing non-persistent messages occasionally, and we solved the problem by tweaking the speed to normal. That was a learning experience! For a week, we kept swearing MQ doesn't lose messages, it must be the app, and lo and behold, we came across a situation where MQ could lose a message. So then I started thinking, "Why not make all our channels normal speed?". I don't want to ever go thru that again (MQ losing messages in transit). But how would that effect our channel performance? I started to look at queue statistics to see what the max depth of our XMIT queues was getting to. None of our XMIT queues (other than Batch specific ones) showed that they ever got deeper than like 7 or 8. To me this said that the batch interval was coming into play. The XMIT queue was draining to zero, the batch size of 50 was not reached, so wait BATCHINT before committing. Since our BATCHINT is 0, commit immediately. The channels are so fast that our transmit queues stay empty almost always. Setting the speed to normal would mean that very rarely maybe a non persistent message would have to wait a tiny bit as a batch of 7 or 8 went across. Then the XMIT queue hit zero, and everything got committed. We just don't have the insane volume here where BATCHSIZE ever come into play. My feeling is that in setting the speed to normal, 99.9999% of my message throughput remained the same, but I insured I would never lose a message on a channel. (Channels dedicated to Batch applications are tuned completely differently). So I think that even though my channels are normal speed, all messages are made available on the Receiving QM almost immediately. The real question is how are they handled at that point. -----Original Message----- From: Neil Casey [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 9:57 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Hi... not an IBMer, but want to comment. The following are my thoughts based on behaviour I have seen on my queue managers, not based on the code. Q1. You will see log I/O in your environment, because you are running NPMSpeed normal. This causes the channel sequence numbers to be hardened, which causes queue file (on the SYSTEM.CHANNEL.SYNCQ) and log I/O. It also causes non-persistent messages to be written inside syncpoint control by the channel agents, so you can't get the performance advantages of direct message copying between the agents. I have recently come to understand that MQ also uses scratchpad files to track channel info, and there is I/O to these files as well. If you were to change to NPMSPEED(fast), then you should reduce both log and queue file I/O in case 1. However, when you have persistent and non-persistent messages sharing an NPMSpeed(Fast) channel, each persistent message will start a batch, and non-persistent messages can get sent in the batch, and so the non-persistent messages get caught up in the synpoints (but their data is not logged). This delays their availability on the target queue. Queue file I/O should generally asynchronous (look on it like paging to virtual memory) whereas logging writes are (at least sometimes) synchronous. (Data Logging is probably async, but syncpoint logging has to be synchronous I/O). Think of it just like a database logger. This means that not all I/O is equally bad. I would suggest that to get maximum performance, you need to separate your persistent and non-persistent traffic, and run NPMSpeed(Fast). This is difficult, as you need to double your infrastructure, and you won't be able to use default transmission queues (I have always thought they were a bad idea, but that is just me). You define 2 channels from each leaf to the hub, and from the hub to each leaf, one for persistent and one for non-persistent. You need to have QMgrAliases everywhere, with different names for accesses to a queue manager with persistent vs non-persistent messages. You need to have additional ReplyToQ remote queue definitions so that replies come back via the correct aliases, as otherwise all of the replies will be routed via the same channel. You also need to define QRemotes for all of the queues at every queue manager, rather than just on the hub, as default transmission queues won't separate the persistent and non-persistent traffic. If persistent and non-persistent traffic hits the same queues, you will need to define 2 QRemotes, to direct the traffic to the correct transmistion queues. However, I don't think the situation in Q2 is as bad as you assume. Lots of the I/O which a queue manager does is asynchronous, and there are separate processes (and threads) handling different bits, so not everything waits just because 1 thread is blocked on a synchronous log I/O. On Q3, the data will be written to disk, but the writes can be asynchronous. The queue file will grow to contain all the messages unless you have a process removing them as they are added. Regards... Neil Casey |---------+----------------------------> | | Gary Ward | | | <[EMAIL PROTECTED]| | | A.NET> | | | Sent by: MQSeries| | | List | | | <[EMAIL PROTECTED]| | | n.AC.AT> | | | | | | | | | 30/05/2003 09:47 | | | Please respond to| | | MQSeries List | | | | |---------+----------------------------> >--------------------------------------------------------------------------- -----------------------------------| | | | To: [EMAIL PROTECTED] | | cc: | | Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages | >--------------------------------------------------------------------------- -----------------------------------| Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment???? -----Original Message----- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to "stop" and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case! Or is it like T.Rob suggested: Nonpersistent gets written to disk, persistent gets written to disk AND log, for a double I/O??? -----Original Message----- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 1:19 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Let me throw in my $.02 - which is actually less in Euros lately ;) Q1. There "could" be I/O to the queue filesystem... see more below Q2. The non-persistent messages will not be "affected" by the persistent ones as long as you are using NPMSPEED(FAST) on your channels. They do not wait for a syncpoint. Paul Clarke could further discuss this with you I'm sure! Consider using channel pipelining as well. Q3. Related to Q1, yes you will eventually see I/O under certain circumstances... The certain circumstances have to do with arrival and processing rates of your non-persistent messages along with the setting of your DefaultQBufferSize. Each queue by default only has 64K of memory for non-persistent messages. So, if you're flooded with data or in the case of your 1GB of non-persistent data coming through, I'm sure you had some spill to the disk filesystem. You can augment the DefaultQBufferSize by setting it in the qm.ini in the TuningParameters stanza. HOWEVER, be sure to do this correctly!!! You need to set the parameter, restart the QM, define your queues that should have the extra large buffer, remove the stanza entry, and then restart the QM. You don't want to leave this set for all queues being created! The max size for the buffer is 1MB. Details about this parameter and usage can be found in the various performance report SupportPacs. Also, T.Rob was correct about in-memory transfer, but this only occurs between applications putting and getting from a queue simulataneously. The messages MUST NOT be IN_SYNCPOINT and they must be non-persistent. The applications will directly exchange the message in-memory and satisfy the MQ API calls on both ends (the PUT and the GET). Hope this helps! Gary ------------------------------ Gary J. Ward Senior Consulting Engineer Information Design, Inc. A Premier IBM Business Partner -------- Original Message -------- ==> From: "Wyatt, T. Rob" <[EMAIL PROTECTED]> ==> Date: Thu, 29 May 2003 11:28:59 -0500 Peter, I remember seeing a response from one of the IBM folks who said that some non-persistent message exchanges happen as an in-memory transfer. In general though, assume your non-persistent messages are written to disk but not logged. Persistent messages are written to disk AND to the log. -- T.Rob -----Original Message----- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 11:43 AM To: [EMAIL PROTECTED] Subject: How a MQSeries Hub does its thing with persistent / non-persist m essages Imagine if you will a Hub and Spoke architecture. QMHUB sits in the middle on a 2 CPU server. 1.3gig CPU, 2 gig RAM MQVersion 5.2.1 CSD05 QM1....QM20 are spokes. On QMHUB, there are no application queues. Only queue manager aliases and XMIT queues. There is a QMAlias for QM1 called QM1, and it directs messages to QM1.XMITQ, which is serviced by the SNDR channel off to QM1. This QMAlias / XMITQ / SNDR setup is present for every spoke. And there is also a RCVR channel as well from every spoke. On day 1 there is only non persistent message traffic for all the spokes. On day 2 a pair of the spokes starts exchanging persistent messages. Q1. On day 1, is there any data being written to disk by QMHUB as the messages fly thru? I assume no, since they are not persistent (but see Q3 below). Q2. On day 2, even though we have 2 CPUs, we still have only 1 QM, so I assume all the non persistent messages throughput must be affected by the persistent messages. My reasoning is, as the persistent messages go in and out of the QMAliases, and in and out of the XMIT queues, it has to "stop" and log, right? And if it has to stop and log, then it can't be handling the non persistent ones at the same time right? They have to wait? Q3. I then defined a local queue on QMHUB and used one of the spoke QMs to send non-persistent message to it. 1 GIG worth actually. Now these are not written to disk, cause they are not persistent, so where are they, in memory? I see the queue file grew by over a GIG, so doesn't that mean they are on disk, even though they are non persistent? More details.............. Batch Interval = 0 Batch Size = 50 Non Persistent Message Speed = Normal There is an MQCluster involved, which is why we have the QMAliases (can't cluster XMIT queues). I don't think this effects any of the answers to the above, but I can expound if need be. Peter Potkay MQSeries Specialist The Hartford Financial Services [EMAIL PROTECTED] x77906 IBM MQSeries Certified This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return email and delete this communication and destroy all copies. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive . Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
