That's a very good point. You need to take into account how much virtual storage is available when running parallel sorts in the same address space. You could limit each sort by passing a DSA value to limit DFSORT's dynamic storage adjustment from allocating more storage than is available. This is also another good reason to run with as few work data sets as possible since they each require virtual storage for control blocks, buffers, etc.
I'm also starting to wonder just how much elapsed time would be saved by using this strategy of having a program read the input and then pass to multiple sorts. With today's world of PAV and high speed devices/channels, is this really going to be much faster than having each read the input? I guess testing will bare that out. This strategy is similar to one we used to see employed with BatchPipes. You'd have one reader job which then passed the records to multiple concurrent jobs through pipes. What I tended to find, is that you end up serializing the reads and the overall result is increased elapsed time. Basically they are all going to read at the pace of the slowest sort. For example you pass records to 4 concurrent sorts. SORT1 will get record 1 and then wait until that's been passed to SORT2, SORT3, SORT4 before it gets record 2. Then it waits again for record 3 and so on and so on. These parallel sorts are probably going to be doing a lot of waiting for input. You can probably minimize this with some sort of large buffer area and wait/post logic but my guess is that will be some complex code. IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu> wrote on 02/12/2010 06:37:47 PM: > [image removed] > > Re: SORTWK files > > Blaicher, Chris > > to: > > IBM-MAIN, > > 02/12/2010 06:38 PM > > Sent by: > > IBM Mainframe Discussion List <IBM-MAIN@bama.ua.edu> > > Please respond to IBM Mainframe Discussion List. > > I just noticed the following in one of your postings. > > "If I want to subtask multiple sorts (same input file)," > > If you are going to run multiple sorts concurrently in the same > address space, things can get interesting. NOTE: You may want to > confirm what I say with your sort vendor. > > Sorts like resources, mainly memory. They also think they are the > only thing running in the address space, or that whatever else is > running will not get more memory once the sort has started. This > means that starting multiple sorts can have a very negative impact > on the sorts started first. > > Running multiple sorts will often times get you an 878 type abend, > especially if your REGION is not REGION=0M. > > If all the sorts are similar in size and characteristics, then maybe > a simple thing as passing the proper parameters in SORTPARM may do > the trick. If you have sorts of different sizes and different > characteristics and run them all with a common set of over-rides, > then some are getting more than they need and others are probably > getting less. > > This is not as simple as it sounds. To put it in perspective, I > have 25 years of experience in trying to get it right. Of course, I > am trying to balance anywhere from 1 to 48 concurrent sorts of > different types and sizes. > > You'll get it to run, but getting it to run the best it can is not easy. > > Chris Blaicher > Phone: 512-340-6154 > Mobile: 512-627-3803 > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO > Search the archives at http://bama.ua.edu/archives/ibm-main.html ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
