Larry,
Sounds good. Also, don't forget about the DFSORT hotline
(dfs...@us.ibm.com) if you have questions or need help debugging a problem
as you develop your solution.
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email: bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu> wrote on 02/15/2010
09:28:44 AM:
> [image removed]
>
> Re: SORTWK files
>
> Larry Crilley
>
> to:
>
> IBM-MAIN
>
> 02/15/2010 09:33 AM
>
> Sent by:
>
> IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu>
>
> Please respond to IBM Mainframe Discussion List.
>
> I would not read a single record and pass to each of the sorts. I will
hope
> to fill multiple "buffers" with many records. For example, why not get 5
> buffers (or more) and each buffer is 1 Meg (or larger) in size. Stash as
> many records as possible in each buffer and have each sort pull the
records
> from these buffer areas. These buffer areas could be above the bar.
I'll
> have to do a little buffer management and know when each sort is done
with
> each buffer, but all in all, not too difficult.
>
> -----Original Message-----
> From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
Behalf
> Of David Betten
> Sent: Friday, February 12, 2010 8:48 PM
> To: IBM-MAIN@bama.ua.edu
> Subject: Re: SORTWK files
>
> That's a very good point. You need to take into account how much virtual
> storage is available when running parallel
> sorts in the same address space. You could limit each sort by passing a
> DSA value to limit DFSORT's dynamic storage
> adjustment from allocating more storage than is available. This is also
> another good reason to run with as few work
> data sets as possible since they each require virtual storage for control
> blocks, buffers, etc.
>
>
> I'm also starting to wonder just how much elapsed time would be saved by
> using this strategy of having a program
> read the input and then pass to multiple sorts. With today's world of
PAV
> and high speed devices/channels, is this
> really going to be much faster than having each read the input? I guess
> testing will bare that out.
>
> This strategy is similar to one we used to see employed with BatchPipes.
> You'd have one reader job which then
> passed the records to multiple concurrent jobs through pipes. What I
> tended to find, is that you end up serializing the reads
> and the overall result is increased elapsed time. Basically they are all
> going to read at the pace of the slowest sort. For
> example you pass records to 4 concurrent sorts. SORT1 will get record 1
> and then wait until that's been passed to SORT2, SORT3,
> SORT4 before it gets record 2. Then it waits again for record 3 and so
on
> and so on. These parallel sorts are probably going to be
> doing a lot of waiting for input. You can probably minimize this with
some
> sort of large buffer area and wait/post logic but my guess is
> that will be some complex code.
>
>
>
>
> IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu> wrote on 02/12/2010
> 06:37:47 PM:
>
> > [image removed]
> >
> > Re: SORTWK files
> >
> > Blaicher, Chris
> >
> > to:
> >
> > IBM-MAIN,
> >
> > 02/12/2010 06:38 PM
> >
> > Sent by:
> >
> > IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu>
> >
> > Please respond to IBM Mainframe Discussion List.
> >
> > I just noticed the following in one of your postings.
> >
> > "If I want to subtask multiple sorts (same input file),"
> >
> > If you are going to run multiple sorts concurrently in the same
> > address space, things can get interesting. NOTE: You may want to
> > confirm what I say with your sort vendor.
> >
> > Sorts like resources, mainly memory. They also think they are the
> > only thing running in the address space, or that whatever else is
> > running will not get more memory once the sort has started. This
> > means that starting multiple sorts can have a very negative impact
> > on the sorts started first.
> >
> > Running multiple sorts will often times get you an 878 type abend,
> > especially if your REGION is not REGION=0M.
> >
> > If all the sorts are similar in size and characteristics, then maybe
> > a simple thing as passing the proper parameters in SORTPARM may do
> > the trick. If you have sorts of different sizes and different
> > characteristics and run them all with a common set of over-rides,
> > then some are getting more than they need and others are probably
> > getting less.
> >
> > This is not as simple as it sounds. To put it in perspective, I
> > have 25 years of experience in trying to get it right. Of course, I
> > am trying to balance anywhere from 1 to 48 concurrent sorts of
> > different types and sizes.
> >
> > You'll get it to run, but getting it to run the best it can is not
easy.
>
> >
> > Chris Blaicher
> > Phone: 512-340-6154
> > Mobile: 512-627-3803
> >
> > ----------------------------------------------------------------------
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> > Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> ----------------------------------------------------------------------
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> ----------------------------------------------------------------------
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
