Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to stop and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case! Or is it like T.Rob suggested: Nonpersistent gets written to disk, persistent gets written to disk AND log, for a double I/O??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 1:19 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Let me throw in my $.02 - which is actually less in Euros lately ;) Q1. There could be I/O to the queue filesystem... see more below Q2. The non-persistent messages will not be affected by the persistent ones as long as you are using NPMSPEED(FAST) on your channels. They do not wait for a syncpoint. Paul Clarke could further discuss this with you I'm sure! Consider using channel pipelining as well. Q3. Related to Q1, yes you will eventually see I/O under certain circumstances... The certain circumstances have
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Peter, As far as I know, if a process is performing I/O, it should not be locking up the CPU. The process will swap out (call it what you will on your favorite OS) while performing I/O and other processes that want to use the CPU are scheduled. I believe this is computing 101... but I took that course a LONG time ago! ;) Regards, Gary -- Gary J. Ward Senior Consulting Engineer Information Design, Inc. A Premier IBM Business Partner Original Message == From: Potkay, Peter M (PLC, IT) [EMAIL PROTECTED] == Date: Fri, 30 May 2003 08:50:12 -0400 Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non- persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non- persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Wien.AC.AT]On Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to stop and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case! Or is it like T.Rob suggested: Nonpersistent gets written to disk, persistent gets written to disk AND log, for a double I/O??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent:
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
This would be true for the smallest unit of execution (normally a thread, not a process). Thus if a thread of execution was committing data to disk, that thread would not continue until the commit was completed. However, another thread within the same process would get CPU if it was able to execute some code. It all depends on whether the MQ processes are multi-threaded or not. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 12:50 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to stop and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case! Or is it like T.Rob suggested: Nonpersistent gets written to disk, persistent gets written to disk AND log, for a double I/O??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 1:19 PM To: [EMAIL
Create Message Set With DTD Problem
Hi, I am trying to create a message set with an XML DTD in MQSI v2.1. When I import the DTD, the system (my PC) takes a long time to responce where I checked the Windows task manager having a 100% CPU usage. After a long time, MQSI returns a message said my newly created message set "locked" which is not true since I checked in and out after I create the message set before import the DTD. Does anyone know what's wrong here?Thanks Mike Do you Yahoo!? Free online calendar with sync to Outlook(TM).
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Neil, I am specifically thinking of the scenario where the message is already on the destination QM. Regardless of whether the messages were in syncpoint or not as they traveled across the channel, I am only concerned about how messages are handled in a HUB queue manager once they are committed to the HUB queue manager. On a busy HUB, at any given moment, I will have thousands of messages being made available every minute to the QM. At the risk of hijacking this thread into a discussion on channel speed: I chose a speed of normal on my channels because for the longest time, we had a batch interval of zero and a batch size of 50 for all our channels. Then one of our channels started losing non-persistent messages occasionally, and we solved the problem by tweaking the speed to normal. That was a learning experience! For a week, we kept swearing MQ doesn't lose messages, it must be the app, and lo and behold, we came across a situation where MQ could lose a message. So then I started thinking, Why not make all our channels normal speed?. I don't want to ever go thru that again (MQ losing messages in transit). But how would that effect our channel performance? I started to look at queue statistics to see what the max depth of our XMIT queues was getting to. None of our XMIT queues (other than Batch specific ones) showed that they ever got deeper than like 7 or 8. To me this said that the batch interval was coming into play. The XMIT queue was draining to zero, the batch size of 50 was not reached, so wait BATCHINT before committing. Since our BATCHINT is 0, commit immediately. The channels are so fast that our transmit queues stay empty almost always. Setting the speed to normal would mean that very rarely maybe a non persistent message would have to wait a tiny bit as a batch of 7 or 8 went across. Then the XMIT queue hit zero, and everything got committed. We just don't have the insane volume here where BATCHSIZE ever come into play. My feeling is that in setting the speed to normal, 99.% of my message throughput remained the same, but I insured I would never lose a message on a channel. (Channels dedicated to Batch applications are tuned completely differently). So I think that even though my channels are normal speed, all messages are made available on the Receiving QM almost immediately. The real question is how are they handled at that point. -Original Message- From: Neil Casey [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 9:57 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Hi... not an IBMer, but want to comment. The following are my thoughts based on behaviour I have seen on my queue managers, not based on the code. Q1. You will see log I/O in your environment, because you are running NPMSpeed normal. This causes the channel sequence numbers to be hardened, which causes queue file (on the SYSTEM.CHANNEL.SYNCQ) and log I/O. It also causes non-persistent messages to be written inside syncpoint control by the channel agents, so you can't get the performance advantages of direct message copying between the agents. I have recently come to understand that MQ also uses scratchpad files to track channel info, and there is I/O to these files as well. If you were to change to NPMSPEED(fast), then you should reduce both log and queue file I/O in case 1. However, when you have persistent and non-persistent messages sharing an NPMSpeed(Fast) channel, each persistent message will start a batch, and non-persistent messages can get sent in the batch, and so the non-persistent messages get caught up in the synpoints (but their data is not logged). This delays their availability on the target queue. Queue file I/O should generally asynchronous (look on it like paging to virtual memory) whereas logging writes are (at least sometimes) synchronous. (Data Logging is probably async, but syncpoint logging has to be synchronous I/O). Think of it just like a database logger. This means that not all I/O is equally bad. I would suggest that to get maximum performance, you need to separate your persistent and non-persistent traffic, and run NPMSpeed(Fast). This is difficult, as you need to double your infrastructure, and you won't be able to use default transmission queues (I have always thought they were a bad idea, but that is just me). You define 2 channels from each leaf to the hub, and from the hub to each leaf, one for persistent and one for non-persistent. You need to have QMgrAliases everywhere, with different names for accesses to a queue manager with persistent vs non-persistent messages. You need to have additional ReplyToQ remote queue definitions so that replies come back via the correct aliases, as otherwise all of the replies will be routed via the same channel. You also need to define QRemotes for all of the queues at every queue manager, rather than just on the hub, as default transmission queues won't
Sharing a MQ Message
Which MQ feature should I look for if my applications would like to share messages. Would like to deliver a message to a specific queue and allow multiple API's to consume that message and not have a message gone until each API has actually consumed a specific message. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Monitor a file and put to Q
We had a similar requirement in our AS/400 applications to send any changed/deleted/added records from certain files as an MQ message. We use the DB2/400 database triggers as the mechanism to initiate the process. Because of the issues involved with database triggers, wewrote a Generic Database Trigger process that is file driven and offers more flexibility than using the DB2/400 db trigger directly. All the application programmer needs to do is turn on the database trigger(s) for the desired files, add a record to a Trigger Control File that contains information about the program to call, andwrite a simple program to Put the MQ message. I would have liked them to use a single generic program to PUT the entire record out as a message, but the different applications had different requirements and wanted to control it themselves, so they write the programs that PUT the messages. I've attached the documentation that we give to the application programmers that describes how to use the Generic Database Trigger process. If you are interested in the source for the RPGLE program, let me know. It contains only the db trigger handling process, not the MQ PUT process. Lynn -Original Message-From: MQSeries List [mailto:[EMAIL PROTECTED]On Behalf Of eai grpSent: Thursday, May 29, 2003 7:06 PMTo: [EMAIL PROTECTED]Subject: Re: Monitor a file and put to Q oh I wish we could ask them to do that, but why would the applications even want to adapt. So from whatyou allsuggest , there is nothing already available and we should write one. Tim Armstrong [EMAIL PROTECTED] wrote: Can you modify the application to generate the MQ message at the time itwrites to the file? Much simpler if you can.RegardsTim Aeai grp<[EMAIL PROTECTED] To: [EMAIL PROTECTED] cc:Sent by: MQSeries Subject: Monitor a file and put to QList<[EMAIL PROTECTED]N.AC.AT30/05/2003 06:52Please respond toMQSeries ListHi All,I have a requirement , where in an application writes to an AS/400 file andI need a tool/software?adapter that monitors this file and writes thismodified line into an MQSeries Queue.Is there something already availableor If I should write one, pls gimme a few tips if I need to Poll or trigger(Performance Issues) etc.Pls SuggestThank In AdvanceDo you Yahoo!?Free online calendar with sync to Outlook(TM).Instructions for managin g your mailing list subscription are provided inthe Listserv General Users Guide available at http://www.lsoft.comArchive: http://vm.akh-wien.ac.at/MQSeries.archive Do you Yahoo!?Free online calendar with sync to Outlook(TM). IFDBTRG_GenericDBTrigger.doc Description: MS-Word document
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
So here is my real question, which is what makes me wonder exactly how a QM handles messages. Our HUB server is using Veritas. The disk that is being written to (whenever that may be) is actually on the Storage Area Network (SAN). The HUB is also clustered with 2 queue managers dedicated to MQSI. The HUB acts simply as a gateway queue manager for this MQSI cluster. THE MQSI boxes are in 2 separate locations, with Veritas, and both again also write to the SAN. Whenever we make bin changes to the SAN, that change ripples across the fabric, making the SAN unavailable for a tiny bit of time. Now, we have an app that is counting milliseconds in its roundtrip of the message. This message starts on one of the spokes, comes to the HUB, is round robined to one of the MQSI boxes, the processed message comes back to the HUB, which then sends it down to the receiving spoke. It processes the message, sends it back to the hub, round robined into MQSI for processing the reply, the processed reply goes back to the hub, which then sends the reply back to the originating spoke. For 99.99% of the time, this roundtrip takes under 500 milliseconds. The app waits up to 2000 milliseconds for the reply. The messages are non persistent and about 25K in size. Whenever the bin changes to the SAN take place, we start getting messages that take longer than 2000 milliseconds, and now we have orphaned replies. These are non persistent messages that are under 64K, so why does a change that makes the disk unavailable cause these messages to slow down? My guess is that the persistent messages the HUB is processing at the same time (or the 64K Nonpersistent ones) must somehow be effecting the performance of the non persistent ones. And I also assume that channel speed has nothing to do with this. So the angle I am after here is how can I increase the performance of my messages for this app so that changes to the SAN don't effect it. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 10:18 AM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages This would be true for the smallest unit of execution (normally a thread, not a process). Thus if a thread of execution was committing data to disk, that thread would not continue until the commit was completed. However, another thread within the same process would get CPU if it was able to execute some code. It all depends on whether the MQ processes are multi-threaded or not. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 12:50 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want
Re: Security exit MQCD fields
Thanks David and Marty! The exit is a LOT more useful now. :-) -- T.Rob -Original Message- From: David C. Partridge [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 5:05 AM To: [EMAIL PROTECTED] Subject: Re: Security exit MQCD fields The first time you will see all the relevant fields completed in the MQCD and MQCXP is when the exit is driven with either MQXR_INIT_SEC or MQXR_SEC_MSG. In a typical scenario, there will be a channel security exit at both ends of the channel, and the client end will be driven with INIT_SEC and will have sent a security message. In this case, the next event at the server end will be a call to the security exit with reason MQXR_SEC_MSG. If there is no security exit at the client or the client responded MQXCC_OK to MQXR_INIT_SEC, then the next event at the server will be a call to the security exit with reason MQXR_INIT_SEC. In either case this is the point at which you would expect things like to MQCXP.PartnerName to be filled in. In fact the manual says of this field: When the exit is initialized this field is blank because the queue manager does not know the name of the partner until after initial negotiation has taken place. Regards, David C. Partridge Security and MQ Products Manager Primeur Group Tel: +44 (0)1926 511058 Mobile: +44 (0)7713 880197 Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
CASE statement problem....
Hi All I have been stuck on a syntax error for some time so I figured I would reach out for a bit of help on this one:) Environment: WMQI 2.1 CSD3WMQ 5.3 CSD 1 I am trying to code a CASE statement in a compute node but am getting a syntax error. I have looked at it in the ESQL reference and even tried a sample from there with no luck. Does anyone have a code snippet of a working CASE statement?!? I feel like I have tried every different combination of CASE - WHEN - THEN with no luck. There must be 1 more combination out there that is actually the correct one:) Thanks Dan Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Compiling shared libraries
It has been a while since I compiled on Linux, so you may want to check the help. I assume you are using GNU C++ compiler so try doing gcc --help I believe. If my guess it right it is '-G' or similar. Hope this helps. Roger, since you have a Linux system could you help this fellow ;-) Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 7:30 AM To: [EMAIL PROTECTED] Subject: Compiling shared libraries Howdy all, I am porting some C++ MQ code from Win32 to Linux and have never used shared libraries. While I can comile my MQ code, I cannot figure out how to link the shared libraries (static libraries are ok). Can anyone give me some usefull pointers ? Sid Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
MQ behaviour - persistent messages availability when the qmgr crashes
Hello there. When an application puts a persistent msg on a queue and if the corresponding qmgr fails before the messge is committed/backed out . can the message still be recovered after the qmgr comes up ? I mean, is the message logged ? Does a qmgr perform a commit/backout just before it crashes, if it is the transaction manager ? Forgive me if the questions sound silly. thnx. Diwakar. American Express made the following annotations on 05/30/2003 07:58:49 AM -- ** This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you. ** == Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Can you send high priority messages down their own channels and the persistent ones down their own. These would run as separate processes and (possibly) not block each other. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 14:21 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages So here is my real question, which is what makes me wonder exactly how a QM handles messages. Our HUB server is using Veritas. The disk that is being written to (whenever that may be) is actually on the Storage Area Network (SAN). The HUB is also clustered with 2 queue managers dedicated to MQSI. The HUB acts simply as a gateway queue manager for this MQSI cluster. THE MQSI boxes are in 2 separate locations, with Veritas, and both again also write to the SAN. Whenever we make bin changes to the SAN, that change ripples across the fabric, making the SAN unavailable for a tiny bit of time. Now, we have an app that is counting milliseconds in its roundtrip of the message. This message starts on one of the spokes, comes to the HUB, is round robined to one of the MQSI boxes, the processed message comes back to the HUB, which then sends it down to the receiving spoke. It processes the message, sends it back to the hub, round robined into MQSI for processing the reply, the processed reply goes back to the hub, which then sends the reply back to the originating spoke. For 99.99% of the time, this roundtrip takes under 500 milliseconds. The app waits up to 2000 milliseconds for the reply. The messages are non persistent and about 25K in size. Whenever the bin changes to the SAN take place, we start getting messages that take longer than 2000 milliseconds, and now we have orphaned replies. These are non persistent messages that are under 64K, so why does a change that makes the disk unavailable cause these messages to slow down? My guess is that the persistent messages the HUB is processing at the same time (or the 64K Nonpersistent ones) must somehow be effecting the performance of the non persistent ones. And I also assume that channel speed has nothing to do with this. So the angle I am after here is how can I increase the performance of my messages for this app so that changes to the SAN don't effect it. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 10:18 AM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages This would be true for the smallest unit of execution (normally a thread, not a process). Thus if a thread of execution was committing data to disk, that thread would not continue until the commit was completed. However, another thread within the same process would get CPU if it was able to execute some code. It all depends on whether the MQ processes are multi-threaded or not. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 12:50 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your
Re: Compiling shared libraries
Try linking with the gcc -G option (gcc -G module1.o module2.o etc.) I think that creates shared libraries on Linux. Try gcc --help or search the web fore details on gcc. Regards John. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 11:30 To: [EMAIL PROTECTED] Subject: Compiling shared libraries Howdy all, I am porting some C++ MQ code from Win32 to Linux and have never used shared libraries. While I can comile my MQ code, I cannot figure out how to link the shared libraries (static libraries are ok). Can anyone give me some usefull pointers ? Sid Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive ** Click here to visit the Argos home page http://www.argos.co.uk The information contained in this message or any of its attachments may be privileged and confidential, and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you are not the intended addressee, any disclosure, reproduction, distribution, dissemination or use of this communication is not authorised. If you have received this message in error, please advise the sender by using your reply facility in your e-mail software. All messages sent and received by Argos Ltd are monitored for virus, high risk file extensions, and inappropriate content. As a result users should be aware that mail maybe accessed. ** Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
The HUB has dozens of channels to and from each spoke. My question is if one pair of spokes is exchanging Nonpersistent messages and another pair starts sending persistent, will they hurt each other. I don't think dedicating channels to be persistent or not between a spoke QM and the HUB will make a difference, since either way, the HUB QM has to deal with dozens of channels either way. It may make a difference on how fast a message gets from a particular spoke to the HUB, but not what happens once it is already there. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 12:00 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Can you send high priority messages down their own channels and the persistent ones down their own. These would run as separate processes and (possibly) not block each other. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 14:21 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages So here is my real question, which is what makes me wonder exactly how a QM handles messages. Our HUB server is using Veritas. The disk that is being written to (whenever that may be) is actually on the Storage Area Network (SAN). The HUB is also clustered with 2 queue managers dedicated to MQSI. The HUB acts simply as a gateway queue manager for this MQSI cluster. THE MQSI boxes are in 2 separate locations, with Veritas, and both again also write to the SAN. Whenever we make bin changes to the SAN, that change ripples across the fabric, making the SAN unavailable for a tiny bit of time. Now, we have an app that is counting milliseconds in its roundtrip of the message. This message starts on one of the spokes, comes to the HUB, is round robined to one of the MQSI boxes, the processed message comes back to the HUB, which then sends it down to the receiving spoke. It processes the message, sends it back to the hub, round robined into MQSI for processing the reply, the processed reply goes back to the hub, which then sends the reply back to the originating spoke. For 99.99% of the time, this roundtrip takes under 500 milliseconds. The app waits up to 2000 milliseconds for the reply. The messages are non persistent and about 25K in size. Whenever the bin changes to the SAN take place, we start getting messages that take longer than 2000 milliseconds, and now we have orphaned replies. These are non persistent messages that are under 64K, so why does a change that makes the disk unavailable cause these messages to slow down? My guess is that the persistent messages the HUB is processing at the same time (or the 64K Nonpersistent ones) must somehow be effecting the performance of the non persistent ones. And I also assume that channel speed has nothing to do with this. So the angle I am after here is how can I increase the performance of my messages for this app so that changes to the SAN don't effect it. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 10:18 AM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages This would be true for the smallest unit of execution (normally a thread, not a process). Thus if a thread of execution was committing data to disk, that thread would not continue until the commit was completed. However, another thread within the same process would get CPU if it was able to execute some code. It all depends on whether the MQ processes are multi-threaded or not. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 12:50 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Peter, Got a question. On my arcitectural diagrams I have specifications for 8-WAY servers. I agree with you that a CPU can only do one thing at a time. While that CPU is waiting what are the other 7 doing? If the are also waiting on #1 what is the use of haveing a multi CPU machine except for IBM to charge us capasity units for MQ/MQSI/Websphere. bee-oh-dubble-bee-dubble-egh From: Potkay, Peter M (PLC, IT) [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Date: Fri, 30 May 2003 08:50:12 -0400 Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non-persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non-persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to stop and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case! Or is it like T.Rob suggested: Nonpersistent gets written to disk, persistent gets written to disk AND log, for a double I/O??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent:
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
That because Gary is very old! (tee hee hee) From: Gary Ward [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Date: Fri, 30 May 2003 09:11:49 -0400 Peter, As far as I know, if a process is performing I/O, it should not be locking up the CPU. The process will swap out (call it what you will on your favorite OS) while performing I/O and other processes that want to use the CPU are scheduled. I believe this is computing 101... but I took that course a LONG time ago! ;) Regards, Gary -- Gary J. Ward Senior Consulting Engineer Information Design, Inc. A Premier IBM Business Partner Original Message == From: Potkay, Peter M (PLC, IT) [EMAIL PROTECTED] == Date: Fri, 30 May 2003 08:50:12 -0400 Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02... Right about my answer to Q2... I guess I missed your point. Having read your follow-up, I would offer this theory. I think the queue manager MUST be able to do more than a single task at once. That's why it's made up of many individual processes which have dedicated tasks. I would think at any one moment there's a bunch of messages floating around in various states. If the messages are non- persistent, they're not logged and hence can be available immediately (if they're not IN SYNCPOINT with persistent messages). Non- persistent messages only get written to disk when there's not enough memory available to hold them on an individual queue. I'm not sure about persistent messages always going right to the queue file system, but my gut feeling is that they do. If your non-persistent messages are 100K and you have your queues set at the 64KB default, I'm pretty sure they go right to disk. That's why you should tune that non-persistent message buffer. Hopefully something is already waiting for these non-persistent messages so they get handed off directly in the manner that T. Rob and I mentioned earlier in this thread. Then there's no I/O at all. Hope this helps... any IBMer's want to comment -Original Message- From: MQSeries List [mailto:[EMAIL PROTECTED] Wien.AC.AT]On Behalf Of Potkay, Peter M (PLC, IT) Sent: Thursday, May 29, 2003 6:11 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi st m essages Gary, I think you answer to Q2 pertains more to when a message is about to leave a server to go onto the next one. That setting tells the receiving side how soon it can have the Nonpersistent message in relation to the batch of messages coming across the channel. I am curious about once the messages have already been accepted on the HUB. Whether they were persistent or not, regardless of the channel speed setting, at any given moment on our busy HUB, the QM finds itself with lots and lots of messages that it now has complete control over. As it routes them thru the QMAliases and the XMITQs, it has to stop and log the persistent messages. I feel that this activity must somehow also effect the non persistent ones as well, since the QM can only do one thing at a time, regardless how fast it does it. If it is busy logging a persistent message, it can't route a non persistent one at that exact moment, correct? Regarding the queue buffer setting, if my messages are less than 64K, and because due to high activity all my channels in a particular SPOKE-HUB-SPOKE route are running, then a non persistent message would go in and out of each XMIT queue, in and out of each QM Alias queue and in and out of each application queue (assuming the app has an outstanding GET with wait) with no I/O to the disk? What if the messages are 100K non persistent ones and the buffer setting is still at default? Are you saying that a non persistent message is still written to disk? If yes, to me that sounds like there is no reason to not use persistence always on any message larger than 64K. Surely that can't be the case!
Re: CASE statement problem....
Here are some examples. set MailType = CASE InputBody.WfMessage.ActivityImplInvoke.ProgramID.ProgramName when 'WorkItemNotification' then 'is Ready for Your Approval' when 'RejectionNotice' then 'Has Been Disapproved' when 'ReWorkNotice ' then 'Requires Rework' when 'SendEmail' then 'Has Been Approved' else 'is In Process' end; set Environment.Variables.From = CASE InputBody.WfMessage.ActivityImplInvoke.ProgramID.ProgramName when 'WorkItemNotification' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail when 'RejectionNotice' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail when 'ReWorkNotice ' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail when 'SendEmail' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail else InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail end; set Environment.Variables.EmailList = CASE InputBody.WfMessage.ActivityImplInvoke.ProgramID.ProgramName when 'WorkItemNotification' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.ApprovalAssignments[NextIx].UserEmail when 'RejectionNotice' then InputBody.WfMessage.ActivityImplInvoke.ProgramID.OriginatorEmail when 'ReWorkNotice ' then InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.ApprovalAssignments[ReworkIx].UserEmail when 'SendEmail' then InputBody.WfMessage.ActivityImplInvoke.ProgramID.OriginatorEmail else InputBody.WfMessage.ActivityImplInvoke.ProgramInputData.CIPData.UserEmail end; Raul L. Acevedo Consulting IT Architect Enterprise Application Integration (EAI) IBM Global Services 818-539-3203 Office (TL 396-3203) Glendale, CA 818-599-6626 Mobile [EMAIL PROTECTED] Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Sharing a MQ Message
Wesley, You are strattleing across application and MQ functionality. MQ Functionality MQGET with the browse option will let you get a message and not distroy it. now. Application Functionality How do you tell when the last application on the list viewed the message bobbee From: Wesley Shaw [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Sharing a MQ Message Date: Fri, 30 May 2003 10:01:38 -0400 Which MQ feature should I look for if my applications would like to share messages. Would like to deliver a message to a specific queue and allow multiple API's to consume that message and not have a message gone until each API has actually consumed a specific message. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive _ MSN 8 helps eliminate e-mail viruses. Get 2 months FREE*. http://join.msn.com/?page=features/virus Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: MQ behaviour - persistent messages availability when the qmgr crashes
If you application didn't receive the RC=0 I believe it would be considered an inflight transaction and would be backed out upon QMGR restart. From: Diwakar S Yammanuru [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: MQ behaviour - persistent messages availability when the qmgr crashes Date: Fri, 30 May 2003 10:58:39 -0400 Hello there. When an application puts a persistent msg on a queue and if the corresponding qmgr fails before the messge is committed/backed out . can the message still be recovered after the qmgr comes up ? I mean, is the message logged ? Does a qmgr perform a commit/backout just before it crashes, if it is the transaction manager ? Forgive me if the questions sound silly. thnx. Diwakar. American Express made the following annotations on 05/30/2003 07:58:49 AM -- ** This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you. ** == Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive _ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Compiling shared libraries
Here is a makefile for Linux. This makefile compiles test1.c into a shared
library of test1.so
+ cut here ++
# Makefile
CC = cc
CFLAGS = -G -I. -DUNIX
CFLAGS_COMPILE_ONLY = -c -DUNIX
LIBS = -lmqm
#
.c.o:
${CC} ${CFLAGS_COMPILE_ONLY} ${CFLAGS} $
OBJ = test1.o
all: test1.so
test1.so: ${OBJ}
${CC} ${CFLAGS} -G -o $@ ${OBJ} ${LIBS}
+ cut here ++
Make sure when you cut and paste this makefile that the indented lines use
a single tab and NOT spaces.
later
Roger...
At 10:53 AM 5/30/2003, you wrote:
It has been a while since I compiled on Linux, so you may want to check the
help. I assume you are using GNU C++ compiler so try doing gcc --help I
believe. If my guess it right it is '-G' or similar.
Hope this helps.
Roger, since you have a Linux system could you help this fellow ;-)
Chris
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Friday, May 30, 2003 7:30 AM
To: [EMAIL PROTECTED]
Subject: Compiling shared libraries
Howdy all,
I am porting some C++ MQ code from Win32 to Linux and have never used shared
libraries.
While I can comile my MQ code, I cannot figure out how to link the shared
libraries (static libraries are ok).
Can anyone give me some usefull pointers ?
Sid
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: Sharing a MQ Message
A possible candidate for publish/subscribe? -Original Message- From: Wesley Shaw [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 7:02 AM To: [EMAIL PROTECTED] Subject: Sharing a MQ Message Which MQ feature should I look for if my applications would like to share messages. Would like to deliver a message to a specific queue and allow multiple API's to consume that message and not have a message gone until each API has actually consumed a specific message. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: MQ behaviour - persistent messages availability when the qmgr crashes
The message might be logged, but it cannot be recovered. A qmgr crash effects a backout, but I would say it happens more after the fact, rather than just before. -Original Message- From: Diwakar S Yammanuru [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 7:59 AM To: [EMAIL PROTECTED] Subject: MQ behaviour - persistent messages availability when the qmgr crashes Hello there. When an application puts a persistent msg on a queue and if the corresponding qmgr fails before the messge is committed/backed out . can the message still be recovered after the qmgr comes up ? I mean, is the message logged ? Does a qmgr perform a commit/backout just before it crashes, if it is the transaction manager ? Forgive me if the questions sound silly. thnx. Diwakar. American Express made the following annotations on 05/30/2003 07:58:49 AM -- ** This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you. ** == Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: MQ behaviour - persistent messages availability when the qmgr crashes
Here is a situation: An application is running, It reads a Q or 2 writes a Q or 2 all in a UOW and the QMGR goes down now part of that UOW is a PUT to a queue prior to the MQCOMIT to ICE the processing deal. Are the messages comitted when the QMGR comes back up? Mr Diwakar never said he was in the middle of a MQCOMIT or MQBACK he just said BEFORE they were comitted. Granted if you issue a MQCOMIT/MQBACK this may or may not be the case but what are the implications when the QMGR takes a holiday in the middle of your processing (not necessarly in the middle of a direct communication with him/her). bobbee From: John Scott [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: Re: MQ behaviour - persistent messages availability when the qmgr crashes Date: Fri, 30 May 2003 17:10:19 - I think there's a subtle middlepoint here. You may not have received an RC=0 on the commit, but the messages may have been committed anyway. There would be a small window of opportunity where MQ has completed the commit, but before it could return RC=0 to the app, it all goes pear shaped. I think the only guarantee is that either all the messages in the UOW will be committed or all will be backed out. You will never be able to get a situation where some of the messages in a UOW get committed and other in the same UOW get backed out. I've seen other threads talking about this with respect to MQClient and to quote Paul Clarke: The semantics of messaging is exactly the same for a client as for a locally connected application. In other words, in order to do reliable messaging in a local application you must follow a certain set of rules like issuing MQPUTs and MQGETs under transactions etc. These rules are exactly the same for a client. One area which often concerns programmers is the MQCMIT call. What happens if you lose your network half way through an MQCMIT verb and get a MQRC_CONNECTION_BROKEN reason code. Did the transaction commit or didn't it ? Well, this is the same for a local application, you are not guaranteed to get a definitive answer on your transaction commit even for the local application. If you really care, you must do some 'known' action (like put a message to a queue) that you can subsequently check the next time you connect. Regards John. -Original Message- From: Robert Broderick [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 15:28 To: [EMAIL PROTECTED] Subject: Re: MQ behaviour - persistent messages availability when the qmgr crashes If you application didn't receive the RC=0 I believe it would be considered an inflight transaction and would be backed out upon QMGR restart. From: Diwakar S Yammanuru [EMAIL PROTECTED] Reply-To: MQSeries List [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: MQ behaviour - persistent messages availability when the qmgr crashes Date: Fri, 30 May 2003 10:58:39 -0400 Hello there. When an application puts a persistent msg on a queue and if the corresponding qmgr fails before the messge is committed/backed out . can the message still be recovered after the qmgr comes up ? I mean, is the message logged ? Does a qmgr perform a commit/backout just before it crashes, if it is the transaction manager ? Forgive me if the questions sound silly. thnx. Diwakar. American Express made the following annotations on 05/30/2003 07:58:49 AM --- --- *** *** This message and any attachments are solely for the intended recipient and may contain confidential or privileged information. If you are not the intended recipient, any disclosure, copying, use, or distribution of the information included in this message and any attachments is prohibited. If you have received this communication in error, please notify us by reply e-mail and immediately and permanently delete this message and any attachments. Thank you. *** *** === === Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive _ Add photos to your messages with MSN 8. Get 2 months FREE*. http://join.msn.com/?page=features/featuredemail Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive ** Click here to visit the Argos home page http://www.argos.co.uk The information contained in this message or any of its
Re: How a MQSeries Hub does its thing with persistent / non-persistent messages
Peter, If you use NPMSPEED=normal, non-persistent and persistent messages in the same batch all become available at the same time. You can infer from that, that performance of non-persistent messages is dependent on I/O for persistent messages, though I believe it is more likely to be the (synchronous) log I/O rather than (asynchronous) queue I/O. Segregating persistent and non-persistent messages on different channels relieves the above dependency. If one channel is waiting for the log, another should be able to continue transmitting messages until such time as it also needs the log or until the qmgr decides a checkpoint is in order. Bottom line: If your logs are on the SAN, there is a distinct probability that your spoke channels will take a break when that device is not available. my .02 sense (pun intentional). -Original Message- From: Potkay, Peter M (PLC, IT) [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 7:21 AM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages So here is my real question, which is what makes me wonder exactly how a QM handles messages. Our HUB server is using Veritas. The disk that is being written to (whenever that may be) is actually on the Storage Area Network (SAN). The HUB is also clustered with 2 queue managers dedicated to MQSI. The HUB acts simply as a gateway queue manager for this MQSI cluster. THE MQSI boxes are in 2 separate locations, with Veritas, and both again also write to the SAN. Whenever we make bin changes to the SAN, that change ripples across the fabric, making the SAN unavailable for a tiny bit of time. Now, we have an app that is counting milliseconds in its roundtrip of the message. This message starts on one of the spokes, comes to the HUB, is round robined to one of the MQSI boxes, the processed message comes back to the HUB, which then sends it down to the receiving spoke. It processes the message, sends it back to the hub, round robined into MQSI for processing the reply, the processed reply goes back to the hub, which then sends the reply back to the originating spoke. For 99.99% of the time, this roundtrip takes under 500 milliseconds. The app waits up to 2000 milliseconds for the reply. The messages are non persistent and about 25K in size. Whenever the bin changes to the SAN take place, we start getting messages that take longer than 2000 milliseconds, and now we have orphaned replies. These are non persistent messages that are under 64K, so why does a change that makes the disk unavailable cause these messages to slow down? My guess is that the persistent messages the HUB is processing at the same time (or the 64K Nonpersistent ones) must somehow be effecting the performance of the non persistent ones. And I also assume that channel speed has nothing to do with this. So the angle I am after here is how can I increase the performance of my messages for this app so that changes to the SAN don't effect it. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 10:18 AM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages This would be true for the smallest unit of execution (normally a thread, not a process). Thus if a thread of execution was committing data to disk, that thread would not continue until the commit was completed. However, another thread within the same process would get CPU if it was able to execute some code. It all depends on whether the MQ processes are multi-threaded or not. Regards John. -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: 30 May 2003 12:50 To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Gary, About a queue manager, or any process that runs on any computer, I have always assumed, maybe incorrectly, that a CPU on a box can only do one thing at a time. It may be incredibly fast, giving the illusion of many things happening at once, but when you get right down to it, everything has to wait its turn for the CPU to do its thing. And if the CPU is waiting to interact with the disk to write a persistent (or bigger than 64k non-persistent) message (I guess even if that message is going to blink in and out of a QM Alias or XMITQ???) then the CPU cant go on to handle non persistent messages. ??? -Original Message- From: Gary Ward [mailto:[EMAIL PROTECTED] Sent: Thursday, May 29, 2003 7:47 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persistent messages Peter, I'll gladly defer to any IBM MQ Developers lurking on the list since this is getting really 'down and dirty', but in the meantime I'll give you another $.02...
Re: WebSphere Application Server / JMS / MQSeries
Jeff, Can't answer your question directly, but can at least point you to the Info Center in case you haven't already found it: http://www-3.ibm.com/software/webservers/appserv/infocenter.html -- T.Rob -Original Message- From: Jeff A Tressler [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 3:34 PM To: [EMAIL PROTECTED] Subject: WebSphere Application Server / JMS / MQSeries Hi Normal programming tasks use the MQI (MQSeries API) to access MQSeries. Our company is beginning to experiment with WebSphere Application Server and its use of the JMS to access MQSeries. Our Infrastructure Group has set up and configured the system but this group has little or no actual knowledge of MQSeries, WebSphere, or JMS. They basically followed instructions on how to install and configure the components. I have been asked to verify the JMS implementation and test its connectivity to MQSeries. While I know quite a bit about MQSeries my exposure to JMS is limited. How do you verify the installation and configuration of the JMS in WebSphere Application Server? I hope it is as simple as using an equivalent to amqsput. It would be nice to run something like a jmsput and jmsget program to verify the connectivity exists. Thank for your time. Jeff Tressler Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
For Ha-Has, I made a dedicated channel for this app from SPOKE1 to HUBQM. The only messages going over this channel are non persistent. Thousands of messages are zooming across this channel every hour. The XMIT queue never got deeper than 2. The speed is normal. A bin change hits our SAN, which the HUB needs, and the XMIT queue backed up to 22 for a couple of seconds! Since then there have been no changes to the SAN, and the XMIT queue again has not gotten over 2. This to me reinforces that fact that a disk outage on the HUB is effecting non persistent messages somehow. And I am beginning to think there is no way around it. :( About the messages being non persistent / persistent and the channel speed: Even though the messages are non persistent, I still care about them. I have always been of the mind set that whether a message is persistent or not has more to do with how difficult it is for the apps to reproduce the message if it got lost. If it is a big deal, then make it persistent. It will survive anything and eventually be processed. Messages that tend to sit in queues for a long time are susceptible to QMs going down, and thus should be made persistent if they need to survive. The messages in this app are inquiry style. They are invalid 5 seconds after the fact. Even if they were persistent and survived a QM restart, they would still be invalid, so why incur the performance penalties of persistence? Now, that's not to say we don't care if they get lost or not. I always shake my head when I hear people say I made it non persistent because I don't care if it gets lost or not. If you don't care, why did you bother to send it in the first place?!?!? What if MQ was losing 50% of the nonpersistent messages? I couldn't tell the app Hey just resend them, they are only inquiry messages anyway! Nor could I say, Every message in this company is going to be persistent. We don't want to bother with lost messages ever. Its my job to config MQ to be as reliable as possible. An application that sends non persistent inquiry messages that will be invalid in 5 seconds has a reasonable assumption that MQ will do everything it can to deliver them. Just because they don't need to survive a QM restart doesn't mean they are less important. I feel the happy medium between Make all message persistent and Don't expect all your messages to always make it to the other side is to set the message channel speed to normal, as long as conditions warrant it. If you got a BATCHINT of 100 and a BATCHSIZE of 200 and your XMIT queues regularly back up, and the occasional non persistent message is being held back until the batch commits, then no way, the speed should be fast, and live with the fact that it may get lost. But I bet that is not how many of anybody's channels run. I bet most of us have XMIT queues that are normally empty, and the BATCHINT is still set to the default of 0. In this case, setting the speed to normal will have very little effect on overall performance, but will insure that no messages ever get lost. I wonder why IBM choose to have the default setting of the channel speed set to fast? Seems to me it would be better to make the default normal. This would perform just fine for most people and would help MQ's rep of never losing messages. You have no idea what a pain it was discovering that MQ was losing messages over a particular fast channel. Days of blaming the apps with losing the messages, hunting in DLQs all over the place, XMIT queues, application queues, etc. The real kick in the pants is that when a message is lost like this, there is ZERO record of the fact. You are left scratching you head. The man hours wasted on hunting for a message lost like this is just not worth it. I'll gladly take a tiny performance hit in a tiny percentage of the messages I send over an already very fast product. Any people looking to pump up the performance of a channel above and beyond this could then tweak the channel to fast, only after realizing messages could get lost. Maybe when it was time to decide what value to use as a default, the logic was We have a choice of making our product faster out of the box or making our message delivery more assured out of the box. And the choice was to make it fast, in case customers are running performance comparisons against other messaging systems like SONINMQ or MSMQ. Who knows, this is only a guess. -Original Message- From: John Scott [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 1:04 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages I think I joined the thread part way through. Now I'm playing catchup. I've read you original message which I'll add my 2p (English money) in revers(ish) order: Q3. I then defined a local queue on QMHUB and used one of the spoke QMs to send non-persistent message to it. 1 GIG worth actually. Now these are not written to disk, cause they are not persistent,
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Someone please correct me if I am wrong, but what I have seen is that the log files are still written to with non-persistent messages because of the queue manager to queue manager commit/syncpoint activity for messages traveling over the channel. I know this because there is a bug in Windows MQ 5.3 CSD01 that is fixed in CSD03 where a roll-back fails causing our log-files to fill up. Since we are only sending non-persistent messages over these channels, why else would the log files be written to? Maybe this is the same symptom that Peter is seeing. If he had a disk problem would the queue manager wait to write some sync records to the logs? Like I said, I'm not sure if this is true, it's just what I noticed when we had the channel commit bug. Brian -Original Message- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 4:32 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages For Ha-Has, I made a dedicated channel for this app from SPOKE1 to HUBQM. The only messages going over this channel are non persistent. Thousands of messages are zooming across this channel every hour. The XMIT queue never got deeper than 2. The speed is normal. A bin change hits our SAN, which the HUB needs, and the XMIT queue backed up to 22 for a couple of seconds! Since then there have been no changes to the SAN, and the XMIT queue again has not gotten over 2. This to me reinforces that fact that a disk outage on the HUB is effecting non persistent messages somehow. And I am beginning to think there is no way around it. :( About the messages being non persistent / persistent and the channel speed: Even though the messages are non persistent, I still care about them. I have always been of the mind set that whether a message is persistent or not has more to do with how difficult it is for the apps to reproduce the message if it got lost. If it is a big deal, then make it persistent. It will survive anything and eventually be processed. Messages that tend to sit in queues for a long time are susceptible to QMs going down, and thus should be made persistent if they need to survive. The messages in this app are inquiry style. They are invalid 5 seconds after the fact. Even if they were persistent and survived a QM restart, they would still be invalid, so why incur the performance penalties of persistence? Now, that's not to say we don't care if they get lost or not. I always shake my head when I hear people say I made it non persistent because I don't care if it gets lost or not. If you don't care, why did you bother to send it in the first place?!?!? What if MQ was losing 50% of the nonpersistent messages? I couldn't tell the app Hey just resend them, they are only inquiry messages anyway! Nor could I say, Every message in this company is going to be persistent. We don't want to bother with lost messages ever. Its my job to config MQ to be as reliable as possible. An application that sends non persistent inquiry messages that will be invalid in 5 seconds has a reasonable assumption that MQ will do everything it can to deliver them. Just because they don't need to survive a QM restart doesn't mean they are less important. I feel the happy medium between Make all message persistent and Don't expect all your messages to always make it to the other side is to set the message channel speed to normal, as long as conditions warrant it. If you got a BATCHINT of 100 and a BATCHSIZE of 200 and your XMIT queues regularly back up, and the occasional non persistent message is being held back until the batch commits, then no way, the speed should be fast, and live with the fact that it may get lost. But I bet that is not how many of anybody's channels run. I bet most of us have XMIT queues that are normally empty, and the BATCHINT is still set to the default of 0. In this case, setting the speed to normal will have very little effect on overall performance, but will insure that no messages ever get lost. I wonder why IBM choose to have the default setting of the channel speed set to fast? Seems to me it would be better to make the default normal. This would perform just fine for most people and would help MQ's rep of never losing messages. You have no idea what a pain it was discovering that MQ was losing messages over a particular fast channel. Days of blaming the apps with losing the messages, hunting in DLQs all over the place, XMIT queues, application queues, etc. The real kick in the pants is that when a message is lost like this, there is ZERO record of the fact. You are left scratching you head. The man hours wasted on hunting for a message lost like this is just not worth it. I'll gladly take a tiny performance hit in a tiny percentage of the messages I send over an already very fast product. Any people looking to pump up the performance of a channel above and beyond this could then tweak the channel to fast,
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
Here's my mindset. Assuming non-persistent messages do not require disk I/O, then they should continue to flow even when your disk I/O sub-system is temporarily unavailable. Since you experiencing something else, there must be conditions under which NP messages are dependent on the disk. I was both attempting to identify some of likely causes and, somewhat-confusingly-in-the-same-breath, suggest that if the logs are on the unavailable disk, you might be heading upstream with a broom stick for a paddle. I stand by my first point, even if there are as few as two messages in a batch. If one is persistent, then non-persistent msgs in the same batch will be blocked until the logs are available. Similarly, the qmgr may at times suspend all activity until it can access the logs. The lesson is to put logs on high-performance, high-availability media--redundant, hot-swapable raid or the like. Putting logs on media that is routinely taken out of service is paramount to drilling holes in your broomstick. I am not necessarily an advocate for separate channels, but I do think it could improve the odds that non-persistent messages will flow when the SAN is unavailable. Generally, while one qmgr process waits for I/O, the others may be dispatched to make good use of the available time, including their own disk I/O and, hopefully, network I/O for non-persistent messages that do not require disk. Tasks waiting on I/O do not consume--they are overlapped with those needing CPU, lest a staggering percentage of the CPU resource would go to waste. The notion of being a CPU being busy waiting is silly--faster machines don't wait at twice the speed! Point #1. If a server is dealing with persistent and non persistent messages, the persistent ones have to negatively effect the non persistent ones, at a hardware level (disk and CPU). It does not matter whether you have separate QMs on that server split between persistent and non persistent. Both QMs share the CPU and disk. It does not matter whether you have separate channels or not. All MCAs share the same CPU and disk. Only true if there is contention for one of the resources: CPU, memory, or I/O. If you can dedicate a task (either by separate Qmgrs or separate channels) to non-persistent messages that are not dependent on disk I/O, then you do not have competion for that resource and there would not be an adverse affect with respect to it. Point #2. If the disk is temporarily unavailable, then persistent messages are effected, and due to point #1, non persistent ones are effected as well. I agree (in a contrary way): If tasks handling non-persistent messages are dependent on disk, then they may be adversely affected by the unavailable disk. If tasks handling non-persistent messages are not dependent on disk, then they may be favorably affected by the unavailable disk! This occurs because other tasks waiting on disk are not competing for CPU. -Original Message- From: Potkay, Peter M (PLC, IT) [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 1:01 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages Hi Dennis. I agree that non-persistent and persistent messages in a batch will be available at the same time. But my batches are of 1 or 2 messages. Since my BATCHINT is zero, and the channel can keep the XMIT queue at zero, I doubt I am incurring any performance hit. 1 or 2 messages are sent across, mybe message #1 is a nonpersistent that is held back until the batch completes, but the batch completes almost immediately, since no more message have yet arrived and the BATCHINT expires. At this point I am still not convinced, in my scenario, that this setting has an effect for the above reason. It has also been mentioned more than once to make a separate channel for persistent vs. non-persistent messages. I don't see how that makes a difference. So I make 2 separate channels from SPOKEQM1 to HUBQM. As far as the hub and its receiving MCAs are concerned, big deal. The HUB already had 50 other receiving MCAs and channels for all the other spokes. If the HUB QM is slowing down because its SAN is temporarily unavailable, does it really matter if you have 1,2 or 20 channels between a particular spoke and the HUB? The HUB still has to deal with the other 50 spokes, and if they are sending persistent messages, then the HUBQM has to service them as well, and it makes no diff that you bothered to separate the channels from any one particular spoke. All these Receiving MCAs have to compete for 1 or 2 CPUs, and if any of those CPUs are busy waiting because the SAN is N/A, then a whole bunch of MCAs are effected. I wonder what everyone thinks of the following: Point #1. If a server is dealing with persistent and non persistent messages, the persistent ones
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
In order to get the behaviour you want, the task processing NP messages must not use or be dependent on SAN I/O whatsoever. Now you don't have absolute control of the I/O that MQ uses. For example, NP messages can spill to disk if they are large or many, and sometimes MQ will use disk scratchpads for various reasons. The good news is that IBM has put a lot of effort into optimizing the throughput of NP messages, so avoids disk I/O unless absolutely necessary. I still think you are probably experiencing log I/O because the channels are doing your NP messages under syncpoint. Change your Ha-Ha channels to NPMSPEED=fast and see if it makes a difference. Ultimately, I think you need to move your logs off the SAN. -Original Message- From: Potkay, Peter M (PLC, IT) [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 2:32 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages For Ha-Has, I made a dedicated channel for this app from SPOKE1 to HUBQM. The only messages going over this channel are non persistent. Thousands of messages are zooming across this channel every hour. The XMIT queue never got deeper than 2. The speed is normal. A bin change hits our SAN, which the HUB needs, and the XMIT queue backed up to 22 for a couple of seconds! Since then there have been no changes to the SAN, and the XMIT queue again has not gotten over 2. This to me reinforces that fact that a disk outage on the HUB is effecting non persistent messages somehow. And I am beginning to think there is no way around it. :( About the messages being non persistent / persistent and the channel speed: Even though the messages are non persistent, I still care about them. I have always been of the mind set that whether a message is persistent or not has more to do with how difficult it is for the apps to reproduce the message if it got lost. If it is a big deal, then make it persistent. It will survive anything and eventually be processed. Messages that tend to sit in queues for a long time are susceptible to QMs going down, and thus should be made persistent if they need to survive. The messages in this app are inquiry style. They are invalid 5 seconds after the fact. Even if they were persistent and survived a QM restart, they would still be invalid, so why incur the performance penalties of persistence? Now, that's not to say we don't care if they get lost or not. I always shake my head when I hear people say I made it non persistent because I don't care if it gets lost or not. If you don't care, why did you bother to send it in the first place?!?!? What if MQ was losing 50% of the nonpersistent messages? I couldn't tell the app Hey just resend them, they are only inquiry messages anyway! Nor could I say, Every message in this company is going to be persistent. We don't want to bother with lost messages ever. Its my job to config MQ to be as reliable as possible. An application that sends non persistent inquiry messages that will be invalid in 5 seconds has a reasonable assumption that MQ will do everything it can to deliver them. Just because they don't need to survive a QM restart doesn't mean they are less important. I feel the happy medium between Make all message persistent and Don't expect all your messages to always make it to the other side is to set the message channel speed to normal, as long as conditions warrant it. If you got a BATCHINT of 100 and a BATCHSIZE of 200 and your XMIT queues regularly back up, and the occasional non persistent message is being held back until the batch commits, then no way, the speed should be fast, and live with the fact that it may get lost. But I bet that is not how many of anybody's channels run. I bet most of us have XMIT queues that are normally empty, and the BATCHINT is still set to the default of 0. In this case, setting the speed to normal will have very little effect on overall performance, but will insure that no messages ever get lost. I wonder why IBM choose to have the default setting of the channel speed set to fast? Seems to me it would be better to make the default normal. This would perform just fine for most people and would help MQ's rep of never losing messages. You have no idea what a pain it was discovering that MQ was losing messages over a particular fast channel. Days of blaming the apps with losing the messages, hunting in DLQs all over the place, XMIT queues, application queues, etc. The real kick in the pants is that when a message is lost like this, there is ZERO record of the fact. You are left scratching you head. The man hours wasted on hunting for a message lost like this is just not worth it. I'll gladly take a tiny performance hit in a tiny percentage of the messages I send over an already very fast product. Any people looking to pump up the performance of a
Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages
With NPMSPEED=fast, you should not lose persistent messages and you should not lose non-persistent messages except when there is somekind of channel abend. Did you experience otherwise? Do you lose more messages with NPMSPEED=fast or from SAN-related timeouts? -Original Message- From: Potkay, Peter M (PLC, IT) [SMTP:[EMAIL PROTECTED] Sent: Friday, May 30, 2003 2:32 PM To: [EMAIL PROTECTED] Subject: Re: How a MQSeries Hub does its thing with persistent / non-persi stent messages For Ha-Has, I made a dedicated channel for this app from SPOKE1 to HUBQM. The only messages going over this channel are non persistent. Thousands of messages are zooming across this channel every hour. The XMIT queue never got deeper than 2. The speed is normal. A bin change hits our SAN, which the HUB needs, and the XMIT queue backed up to 22 for a couple of seconds! Since then there have been no changes to the SAN, and the XMIT queue again has not gotten over 2. This to me reinforces that fact that a disk outage on the HUB is effecting non persistent messages somehow. And I am beginning to think there is no way around it. :( About the messages being non persistent / persistent and the channel speed: Even though the messages are non persistent, I still care about them. I have always been of the mind set that whether a message is persistent or not has more to do with how difficult it is for the apps to reproduce the message if it got lost. If it is a big deal, then make it persistent. It will survive anything and eventually be processed. Messages that tend to sit in queues for a long time are susceptible to QMs going down, and thus should be made persistent if they need to survive. The messages in this app are inquiry style. They are invalid 5 seconds after the fact. Even if they were persistent and survived a QM restart, they would still be invalid, so why incur the performance penalties of persistence? Now, that's not to say we don't care if they get lost or not. I always shake my head when I hear people say I made it non persistent because I don't care if it gets lost or not. If you don't care, why did you bother to send it in the first place?!?!? What if MQ was losing 50% of the nonpersistent messages? I couldn't tell the app Hey just resend them, they are only inquiry messages anyway! Nor could I say, Every message in this company is going to be persistent. We don't want to bother with lost messages ever. Its my job to config MQ to be as reliable as possible. An application that sends non persistent inquiry messages that will be invalid in 5 seconds has a reasonable assumption that MQ will do everything it can to deliver them. Just because they don't need to survive a QM restart doesn't mean they are less important. I feel the happy medium between Make all message persistent and Don't expect all your messages to always make it to the other side is to set the message channel speed to normal, as long as conditions warrant it. If you got a BATCHINT of 100 and a BATCHSIZE of 200 and your XMIT queues regularly back up, and the occasional non persistent message is being held back until the batch commits, then no way, the speed should be fast, and live with the fact that it may get lost. But I bet that is not how many of anybody's channels run. I bet most of us have XMIT queues that are normally empty, and the BATCHINT is still set to the default of 0. In this case, setting the speed to normal will have very little effect on overall performance, but will insure that no messages ever get lost. I wonder why IBM choose to have the default setting of the channel speed set to fast? Seems to me it would be better to make the default normal. This would perform just fine for most people and would help MQ's rep of never losing messages. You have no idea what a pain it was discovering that MQ was losing messages over a particular fast channel. Days of blaming the apps with losing the messages, hunting in DLQs all over the place, XMIT queues, application queues, etc. The real kick in the pants is that when a message is lost like this, there is ZERO record of the fact. You are left scratching you head. The man hours wasted on hunting for a message lost like this is just not worth it. I'll gladly take a tiny performance hit in a tiny percentage of the messages I send over an already very fast product. Any people looking to pump up the performance of a channel above and beyond this could then tweak the channel to fast, only after realizing messages could get lost. Maybe when it was time to decide what value to use as a default, the logic was We have a choice of making our product faster out of the box or making our message delivery more assured out of the box. And the choice was to make it fast, in case customers are running performance comparisons against other messaging systems like SONINMQ or MSMQ. Who knows,
Re: Compiling shared libraries
Howdy John et al,
Thanks to everyone who sent me an email, after playing all night, I came
up with the following two make files. The first builds my shared library,
the second links the MQ C++ shared libraries (and mine) into a test program.
I sugest you file this one away because it is bound to be usefull.
Sid
#-
# Build my various Classes into a shared library
#
CC=g++
TARGET = /opt/mqlink/lib/libmqlink.so
INCLUDE = -I/opt/pgp/headers -I/opt/mqm/inc
OBJECTS = Log.o DataFile.o HL7.o Segment.o MQSupport.o MQAdmin.o BaseXML.o
MD5Hash.o Crypto.o
#DEBUG = -D_DEBUG
DEBUG=
LIBFLAGS= -fPIC -shared -rdynamic
CFLAGS= -fPIC
.cpp.o:
$(CC) $(CFLAGS) -c $(INCLUDE) $(DEBUG) 21 $
all:
$(MAKE) $(TARGET)
$(MAKE) lib
$(TARGET): $(OBJECTS)
lib:
$(CC) $(LIBFLAGS) -o $(TARGET) $(OBJECTS)
clean:
rm -f *.o
[rm -f $(TARGET)
-8
#
# Build our test program and link in the shared libraries
#
# Note: use ldd to confirm whats been linked in.
# Make sure the LD_LIBRARY_PATH is set to include our library.
CC=g++
TARGET=test
OBJS = test.o
LIBS = -ldl -L/opt/mqlink/lib -lmqlink -limqs23gl -limqb23gl
#
# Our Archives (static)
#
ARCS = /opt/pgp/libraries/static/release/libPGPsdk.a
INCLUDE = -I/opt/pgp/headers -I/opt/mqlink/src/commoncode -I/opt/mqm/inc
LDFLAGS=
#DEBUG= -D_DEBUG
DEBUG=
CFLAGS=-fPIC -rdynamic
.cc.o:
$(CC) -c $ $(INCLUDE) $(LIBS) $(ARCS) $(DEBUG) 21
all:
$(MAKE) $(TARGET)
# Compile program,
#
$(TARGET): $(ARCS) $(OBJS) $(LIBS)
$(CC) $(CFLAGS) $(LDFLAGS) $(OBJS) $(LIBS) $(INCLUDE) -o $(TARGET)
21 $
lib:
@cd /opt/mqlink/src/commoncode; if ls *.o /dev/null 21; then
$(MAKE) lib; fi
clean:
@echo Cleaning up
rm -f *.o
rm -f $(TARGET)
--8-
-Original Message-
From: Roger Lacroix [mailto:[EMAIL PROTECTED]
Sent: Saturday, 31 May 2003 2:04 AM
To: [EMAIL PROTECTED]
Subject: Re: Compiling shared libraries
Here is a makefile for Linux. This makefile compiles test1.c into a shared
library of test1.so
+ cut here ++
# Makefile
CC = cc
CFLAGS = -G -I. -DUNIX
CFLAGS_COMPILE_ONLY = -c -DUNIX
LIBS = -lmqm
#
.c.o:
${CC} ${CFLAGS_COMPILE_ONLY} ${CFLAGS} $
OBJ = test1.o
all: test1.so
test1.so: ${OBJ}
${CC} ${CFLAGS} -G -o $@ ${OBJ} ${LIBS}
+ cut here ++
Make sure when you cut and paste this makefile that the indented lines use
a single tab and NOT spaces.
later
Roger...
At 10:53 AM 5/30/2003, you wrote:
It has been a while since I compiled on Linux, so you may want to check the
help. I assume you are using GNU C++ compiler so try doing gcc --help I
believe. If my guess it right it is '-G' or similar.
Hope this helps.
Roger, since you have a Linux system could you help this fellow ;-)
Chris
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
Sent: Friday, May 30, 2003 7:30 AM
To: [EMAIL PROTECTED]
Subject: Compiling shared libraries
Howdy all,
I am porting some C++ MQ code from Win32 to Linux and have never used
shared
libraries.
While I can comile my MQ code, I cannot figure out how to link the shared
libraries (static libraries are ok).
Can anyone give me some usefull pointers ?
Sid
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Re: MQClient thread-safe?
I write multi threaded clients all the time. just make sure you link the correct libraries in and all works fine. Sid -Original Message- From: Stefan Sievert [mailto:[EMAIL PROTECTED] Sent: Saturday, 31 May 2003 5:32 AM To: [EMAIL PROTECTED] Subject: MQClient thread-safe? Happy Friday everybody, can anybody (from IBM?) comment on the following statement: The MQ Client interface is not thread-safe. You cannot safely use MQ Client connections to the same queue manager from multiple threads within the same process (for example, a Workflow Web Client Servlet) ... I recently heard this in a discussion and I am curious to find out if it is accurate. Thanks and a great weekend to y'all, Stefan _ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
