Well, I think that I can work with that. These BLOBs are stored on
tape, each tape represents a day, and we've got 5 months to process.
The amount of data for a day is never more then 4GB - the biggest day
was just short of 3GB. So maybe I could:
- Create a series of pagesets of the maximum size, and define an xmitq
on each one. As I said, we've got disk to burn.
- Have the mainframe loader app write a day's worth of data to each
xmitq queue in turn, either by using a different remote queue each
time, or altering the XMITQ() in the remote queue def each time.
- When the channel to the Solaris machine empties an xmitq queue and
then quiesces, alter the sender to use the next xmitq queue in line.
Or, have a different channel for each xmitq queue, since the order
that the messages arrive isn't important.
- Have a Perl script or something monitoring the Unix filesystem to
make sure it doesn't fill, since the thing that offloads the queue
there is relatively slow. Stop the channel(s) if things are getting
close.
That seems like a decent way to let the fastest part (the mainframe)
get far ahead of the rest of the process, and with only a little
manual intervention. I'm thinking we could do a week at a time, and
when the xmitqs clear, load up the next week. It needs some more
thought, but it sounds promising.
Stefan Raabe
<[EMAIL PROTECTED] To: [EMAIL PROTECTED]
.DE> cc:
Sent by: MQSeries Subject: Re: Really Big Pagesets
List
<[EMAIL PROTECTED]
N.AC.AT>
08/02/03 02:38 AM
Please respond to
MQSeries List
Jim,
maximum pageset size is 4 GB. maximum queue size MQ 5.2 is 2 GB, with
MQ5.3
it is 4 GB.
you may consider to code a special handling for the returncodes "queue
full" and
"pageset full" in your sending program. (e.g. wait some seconds and
retry). or - if the
message sequence does not matter - you could use multiple channels
with
multiple
transmission queues. but in every case you must put some logic into
your
sending application program to "switch" to the other xmit queue (use
another
remoteq).
Regards, Stefan.
Jim Ford wrote:
>In a previous thread, I mentioned that we are sending gobs of BLOBs
>from the mainframe to a Unix machine. The mainframe side will be
>putting the messages far faster than the Solaris side will be
>consuming them. So we want to queue up as much data in the xmitq as
we
>can, and let MQSeries deliver it when it can.
>
>I've got the OS/390 disk available to me to do that (10 3390s), but I
>find that the maximum size of a pageset is 4GB, or a little less than
>2 volumes. I assume that means that a queue cannot contain more than
>4GB of data either.
>
>How can I handle this? I realize that the application could take on
>the burden of knowing when to stop, but we were all hoping to just
>dump the data to MQ and the product unload it over time. It's a rare
>thing here to have this much disk available and I'd like to use it.
>
>Instructions for managing your mailing list subscription are provided
in
>the Listserv General Users Guide available at http://www.lsoft.com
>Archive: http://vm.akh-wien.ac.at/MQSeries.archive
>
>
>
Instructions for managing your mailing list subscription are provided
in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
Instructions for managing your mailing list subscription are provided in
the Listserv General Users Guide available at http://www.lsoft.com
Archive: http://vm.akh-wien.ac.at/MQSeries.archive
