Robert and Peter (and anyone else who is interested).
You are kind of on the right track, some definitions: A "site" is some remote internet connected location, ie a Hospital, surgery, insurance company, health department or private doctor. A "Business Unit" is a unit within our corporation that pumps data (usually medical results) to the QM, results could be pathology, xray reports, accounting data, disease outbreaks etc. A client is the same as a site, where we have our client software running (Win32 only). "Sites" collect encrypted data and use it as required, they can also collect signed XML messages from a queue unique to them. There is a set of common (global) queues that all sites put XML messages and acknowledgments back to (a trigger process gets these and processes them). So every site has 3 queues as defined earlier (a cleartext q, an XML q, and an encrypted data q). The XML data is for things like "What version are you running" ? or can I have a copy of your config file etc... (Command and Control messages). Sites collect from both the XML q and the enc. data q. Site's don't put to the clear text queue unless they are sending data to another site (a feature we are about to offer soon). 1. The load from "clients" is about to grow and must be balanced. - YES 2. You cannot add more NT boxes but can add more Linux boxes. - YES (NT is not a platform we want to grow with - we already have 1100 desktops and 80 server). 3. You cannot process on Linux (neither data from clients nor data from "business units") - Basically correct, all software is Win32 We are deploying new servers as Linux where possible as the company was looking at outsourcing us but we discovered the outsourcer (HP Managed solutions ) has no real Linux support despite what they say in their add's. -----Original Message----- From: Pavel Tolkachev [mailto:[EMAIL PROTECTED] Sent: Sunday, 9 November 2003 6:40 AM To: [EMAIL PROTECTED] Subject: Re: Triggering across New Cluster not working - Design Suggestion Sid, I tried to get the problem and think of the solution, but still have some black spots: i) How do you define "site"? ii) What are the relationships between "client", "business unit", and "site"? Is my understanding of the data flow correct: 1. Business units are connecting from Intranet and pumping data into cleartext queues (ctq), 2. When message arrives to ctq, some application must be triggered and process that message. One of the results of the processing is writing the encrypted message(s) on encrypted queue (eq). Because of the encryption requirement, triggering (or at least data processing) must occur on NT platform 3. Clients from Internet both get and put data (from/on eqs). 4. When a client puts something on eq, another triggering must happen and data must be processed (I assume, because data are encrypted, processing also must be on NT platform)? iii) Is my understanding of your business requirement for the change to clustering correct?: 1. The load from "clients" is about to grow and must be balanced. 2. You cannot add more NT boxes but can add more Linux boxes. 3. You cannot process on Linux (neither data from clients nor data from "business units"). iv) What is the role of XML command queue? I feel I am confused somewhere but cannot even get where ;-) Pavel "can you restate your actual problem from an application perspective?" Peter, In a nut shell.... I have 600+ clients that connect into our queue manager to transfer encrypted medical data. The load will rise to 3000+ over the next 3-12 months as more business units pump data into the QM. I added another QM and clustered them together so that queues could be spread over the cluster and with the intention that round robin DNS would load balance incomming SRVNCON connections over the two boxes. 1. clients connect via the internet and know one name for the cluster (mq.our.host.com). 2. They use SRVCONN channels to connect and technically collect data from any "site" specific queue only. 3. There are 3 queues per site, a cleartext q, an XML command q, and an encrypted data q 4. All my trigger apps are written for NT platform as the crypto libraries are win32 only 5. Business units pump data into the cleartext queue only 6. sites collect their data from the encrypted queue. 7. clients are multi-threaded win32 based (NT4 and up, and can have as many as 5 threads attached to QM). 8. clients connect several times a day for 1-2 minutes only. 9. clients return status and data back to common cluster wide queues for trigger apps to process as required. 10. I have written everything in C++ using the MQ C++ API 11. There is the possibility that several business units will be sold and a queue manager will go with them, hence queues for a business have a common naming convention. That's about it....:) Sid -----Original Message----- From: Potkay, Peter M (PLC, IT) [mailto:[EMAIL PROTECTED] Sent: Saturday, 8 November 2003 2:30 PM To: [EMAIL PROTECTED] Subject: Re: Triggering across New Cluster not working - Design Suggestion Sid, can you restate your actual problem from an application perspective? Just list the requirements of what you are trying to achieve. -----Original Message----- From: Robert Broderick [mailto:[EMAIL PROTECTED] Sent: Friday, November 07, 2003 10:18 PM To: [EMAIL PROTECTED] Subject: Re: Triggering across New Cluster not working - Design Suggestion Isn't this one of the intentions of clustering. To do work load and to some degree failover balancing. If you have more than one server where the message are designed to be routed to you solve your problem of scaling. If one of the destination servers goes down you have automatic faliover to the other members of the cluster. Critical supplies can be made durable by implementing replication and clustering from a hardware perspective. Granted MQSeries in it's self is not an out-of-the box solution for all the problems associated with a messaging system. But then again a hammer alone doesn't build a house. At least you have determined the pit falls of your design at an early stage. With some re-thought and pinging ideas off the LISTSERV you will eventually come up with a system that can take a punch or two. Father Bobbee >From: [EMAIL PROTECTED] >Reply-To: MQSeries List <[EMAIL PROTECTED]> >To: [EMAIL PROTECTED] >Subject: Triggering across New Cluster not working - Design Suggestion >Date: Sat, 8 Nov 2003 08:46:37 +1000 > >Yep, everyone's comments are valid... I re-read the Queue Manger Cluster >manual again last night and found a one liner on the limitation of >MQGET.... >just a single line ( and not a mention in any of the other manuals as far >as >I can see so far). > >I find this a serious limitation for my design. The queue needs some kind >of >flag that says it's available for GETing across the cluster, otherwise you >need to re-design your applications to hunt down the queue. > >I can solve the triggering problem, just trigger locally and have my apps >connect to each server, the real issue is the clients connecting in. > >It does not make sense to me that you can PUT accross the cluster but must >attach locally to the QM to get the data. I now need to do a re-design of >my >client application to obtain it's data. I'm trying to send sensitive >medical >data where a duplication of data could be fatal and with 1000's of >multi-threaded clients coming online in the next 12 months a single server >will not cut it. > >My best solution so far is to have clients connect, discover their queue is >not present (CC=2085), build a Permanent Dynamic queue (or something >similar) and PUT to a Dist list of queues serviced by a data move app to >move the data to the temporary queue. > >Any other design suggestions are more than welcome. > >Sid Young >Brisbane >Australia > > >-----Original Message----- >From: Robert Broderick [mailto:[EMAIL PROTECTED] >Sent: Saturday, 8 November 2003 4:38 AM > >Picking up on what Stefan is indicating. one of the requirements for >triggering is that the INIT Q is available and opened for input by some >process (hopefully the Trigger monitor) from the LINUX side can the QMGR >verify this through the cluster if those attributes are true or false??? > > bobbee > >I will say this is an interesting twist to triggering. MAybe you need a >process on LINUX that starts the NT process like secured shell. > > > >From: Stefan Raabe <[EMAIL PROTECTED]> > >Date: Fri, 7 Nov 2003 11:40:55 +0100 > > > >Sid, > > > >i hope i understood your environment right: unix and nt in an mqseries > >cluster, > >local > >queue on unix should be triggerd, initqueue is a cluster queue that is > >local on > >the > >nt system. > > > >triggering is a "local" kind of processing, which means that the >initiation > >queue has to > >be a local queue. triggering will not work when you specify a cluster >queue > >(which is on a different queuemanager in your case) as initiation queue. > > > >as others already stated, there may be a missunderstanding of the cluster > >functionality.... > >making a queue a cluster queue will make it known within the cluster, but > >it is > >not > >the same as if it is defined in every cluster queuemanager. > > > >these 600 other queues are local on the nt system? if yes, thats why the > >triggering on nt works > >for them. if not, please ignore this mail :-) > > > >regards, stefan > > > > > > > > >Howdy all, > > > >I have just put together a new cluster between two machine (NT4 and >Linux) > > > >The process to run is on the NT machine as well as the trigger monitor >and > >the > >initiation queue that it monitors. I have place a copy of the process > >definition on both machine and the queue with triggering turned on is > >located on > >the linux box. All queues have the cluster defined. > > > >I can see the required queues in the cluster using display qc(*) but >when > >I put > >stuff into that queue (on the linux box) the triggering does not appear >to > >work. > > > >I have 600 other queues still on the NT box which are triggering fine > > > >Any Ideas...Please!!!! > > > > > >Sid Young > >Brisbane > >Australia Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive -- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden. Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive Instructions for managing your mailing list subscription are provided in the Listserv General Users Guide available at http://www.lsoft.com Archive: http://vm.akh-wien.ac.at/MQSeries.archive
