Re: Confused on DFSORT SORTWK

[David Betten]

I think my post was confusing.  Normally you don't need DSNTYPE=LARGE to
use multiple extents.  But in your specific case, you had primary and
secondary of 3335 cylinders (50025 tracks) so a second extent would put you
over the 65535 limit. So that's why I said you needed DSNTYPE=LARGE to use
multiple extents.  Sorry for the confusion.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 05/11/2012
03:13:32 PM:

> From: Scott Barry 
> To: IBM-MAIN@bama.ua.edu,
> Date: 05/11/2012 03:14 PM
> Subject: Re: Confused on DFSORT SORTWK
> Sent by: IBM Mainframe Discussion List 
>
> Why would I be forced to code DSNTYPE=LARGE, rather than using the
> WRKSEC=Y parameter in SORTCNTL along with //SORTWKnn DDs with
> secondary allocation specified?   I am also curious about the SEC=nn
> parameter, as well.
>
> For my interests, I have chosen DSNTYPE=LARGE, however one still
> must remember the 16 extent per volume limitation, so it's oh so
> important to code a sufficiently large enough secondary amount (and/
> or primary as well) so that a one-volume allocation can even exceed
> the 65K track limitation and still be within 16 extents.
>
> Scott Barry
> SBBWorks, Inc.
>
>
> On Tue, 1 May 2012 15:52:41 -0400, David Betten 
wrote:
>
> >Just to clarify, DFSORT can cetrainly use more than the 1st extent, it
just
> >can't use multiple volumes.  As long as DSNTYPE=LARGE is coded on the DD
> >statement, each of the SORTWK DDs should be able to use multiple
extents.
> >Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to
> >get 3 extents of 3335 cylinders on each work data set if the volumes are
> >empty.
> >
> >> From: Neil Duffee 
> >> To: IBM-MAIN@bama.ua.edu,
> >> Date: 05/01/2012 03:14 PM
> >> Subject: Re: Confused on DFSORT SORTWK
> >> Sent by: IBM Mainframe Discussion List 
> >>
> >> In addition to all the other responses you got regarding DynAlloc, I
> >> wanted to add the nudge that DFSORT only uses the **1st** extent in
> >> SortWk DD statements.  Thus, SPACE=(CYL,(3335,3335)) is the same as
> >> SPACE=(CYL,3335) which, in your example, only got you 10*'3335
> >> cylinders or the equivalent to 10 mod-3s.  So you'd need to have 27
> >> DD statements to use your entire pool.
> >>
> >> >  signature = 6 lines follows  <
> >> Neil Duffee, Joe Sysprog, uOttawa, Ottawa, Ont, Canada
> >> telephone:1 613 562 5800 x4585  fax:1 613 562 5161
> >> mailto:NDuffee of uOttawa.ca http:/ /aix1.uOttawa.ca/ ~nduffee
> >> "How *do* you plan for something like that?"  Guardian Bob, Reboot
> >> "For every action, there is an equal and opposite criticism."
> >> "Systems Programming: Guilty, until proven innocent"  John Norgauer
2004
> >>
> >>
> >> -Original Message-
> >> From: Mark Pace [mailto:pacemainl...@gmail.com]
> >> Sent: April 27, 2012 08:43
> >> Subject: Confused on DFSORT SORTWK
> >>
> >> Trying to sort a fairly large RMF data file. 20,000 + cylinders.
> >> I keep running out of SORTWK space.  I have 9 3390-9 volumes in my
> >> SMS temporary storage pool. But I just noticed that no matter how
> >> many SORTWKxx DDs I add it only allocates file on the first 5 temp
> >> storage volumes.
> >> Anyone have an idea how I make DFSORT use more volumes?
> >>
> >> //SORTWK01 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(3335,3335))
> >> [snip]
> >>
> >> ICE046A E SORT CAPACITY EXCEEDED - RECORD COUNT 2459859
> >> [snip]
> >>
> >> --
> >> For IBM-MAIN subscribe / signoff / archive access instructions,
> >> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
> >>
> >
> >
> >Have a nice day,
> >Dave Betten
> >DFSORT Development, Performance Lead
> >IBM Corporation
> >email:  bet...@us.ibm.com
> >1-301-240-3809
> >DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
> >--
> >For IBM-MAIN subscribe / signoff / archive access instructions,
> >send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Confused on DFSORT SORTWK

[David Betten]

DFSORT should be allocating as large format.  Do you have VIO=YES as your
default.  That might be preventing DFSORT from allocating as large format.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 05/06/2012
02:55:53 PM:

> From: Skip Robinson 
> To: IBM-MAIN@bama.ua.edu,
> Date: 05/06/2012 02:57 PM
> Subject: Re: Confused on DFSORT SORTWK
> Sent by: IBM Mainframe Discussion List 
>
> (Reconstructing thread from archive)
>
> I read the posts in this thread but deleted them because they were not
> immediately pertinent to our shop. Then last night I got woke up with a
> similar problem. The default DYNALLOC setting of 3 was insufficient to
> complete a sort involving an unusually large input. We got
>
> ICE083A D RESOURCES WERE UNAVAILABLE FOR DYNAMIC ALLOCATION OF WORK DATA
> SETS (064K)  <-
> ICE753I FWK=(0,0) SWK=(0,0) TWK=(0,0) RWK=(0,0) TOTAL=(0,0) BLK=56600
> ICE248I ATTEMPTED TO ALLOCATE 19389MB OF DISK WORK SPACE ON 3 WORK DATA
> SETS
> ICE249I SUCCESSFULLY ALLOCATED 0MB OF DISK WORK SPACE ON 0 WORK DATA SETS

>
>
> (Last line is probably the world's most disappointing 'success' message
> ever.) In researching the problem, I read that DFSORT for some time has
> been capable of using large or extended format data sets so that the 64K
> limit flagged in ICE083A does not apply. What is required to induce
DFSORT
> to use modern work data sets for dynamic allocation?
>
> .
> .
> JO.Skip Robinson
> SCE Infrastructure Technology Services
> Electric Dragon Team Paddler
> SHARE MVS Program Co-Manager
> 626-302-7535 Office
> 323-715-0595 Mobile
> jo.skip.robin...@sce.com
>
>
>
>
>
>
>
>
>
>
>
>
> Just to clarify, DFSORT can cetrainly use more than the 1st extent, it
> just
> can't use multiple volumes.  As long as DSNTYPE=LARGE is coded on the DD
> statement, each of the SORTWK DDs should be able to use multiple extents.
> Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to
> get 3 extents of 3335 cylinders on each work data set if the volumes are
> empty.
>
> > From: Neil Duffee 
> > To: IBM-MAIN@bama.ua.edu,
> > Date: 05/01/2012 03:14 PM
> > Subject: Re: Confused on DFSORT SORTWK
> > Sent by: IBM Mainframe Discussion List 
> >
> > In addition to all the other responses you got regarding DynAlloc, I
> > wanted to add the nudge that DFSORT only uses the **1st** extent in
> > SortWk DD statements.  Thus, SPACE=(CYL,(3335,3335)) is the same as
> > SPACE=(CYL,3335) which, in your example, only got you 10*'3335
> > cylinders or the equivalent to 10 mod-3s.  So you'd need to have 27
> > DD statements to use your entire pool.
> >
>
>
>
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: SELECTING SMF RECORDS FOR 1 DAY

[David Betten]

Try this in your SYSIN

//SYSIN    DD *
   INDD(INDD1,OPTIONS(DUMP))
   OUTDD(OUTDD1,TYPE(14:15))
   DATE(2012125,2012125)
/*


In my example, the job would select only records from today.  Change the
125 to the day you need to select.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 05/04/2012
07:13:28 AM:

> From: John Dawes 
> To: IBM-MAIN@bama.ua.edu,
> Date: 05/04/2012 07:13 AM
> Subject: SELECTING SMF RECORDS FOR 1 DAY
> Sent by: IBM Mainframe Discussion List 
>
> G'Day,
>
> I am following up on a problem regarding the corruption of a dsn.  I
> have the SMF records for the day, however since the daily tapes are
> merged into a weekly tape I am having a problem (space issues)
> trying to extract only the type 14 & 15 records for that particular
> date.  I looked at - OSMVS System Management Facilities - the doc
> for an example but I didn't find any.  Can someone suggest what I
> can try.  Below is my job which I am trying to run.
>   //STEP1    EXEC PGM=IFASMFDP
> //INDD1    DD  DSN=SYS2.SMFDATA.WEEK.G1584V00,DISP=SHR
> //OUTDD1   DD  DSN=&&SM11,
> //    DISP=(,PASS),UNIT=SYSDA,SPACE=(CYL,(150,150),RLSE),
> //    DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=4096)
> //SYSPRINT DD SYSOUT=*
> //SYSIN    DD *
>    INDD(INDD1,OPTIONS(DUMP))
>    OUTDD(OUTDD1,TYPE(14:15))
> /*
> //STEP2   EXEC PGM=IDCAMS,TIME=1440
> //DD1  DD DSN=&&SM11,
> //    DISP=(OLD,DELETE),
> //    DCB=(RECFM=VBS,LRECL=32760,BLKSIZE=4096)
> //SYSOUT   DD SYSOUT=*
> //SYSPRINT DD SYSOUT=*
> //SYSIN    DD *
>   PRINT INFILE(DD1) DUMP
> /*
> //
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Confused on DFSORT SORTWK

[David Betten]

Just to clarify, DFSORT can cetrainly use more than the 1st extent, it just
can't use multiple volumes.  As long as DSNTYPE=LARGE is coded on the DD
statement, each of the SORTWK DDs should be able to use multiple extents.
Since a 3390-9 has a capacity of 10,017 cylinders, you should be able to
get 3 extents of 3335 cylinders on each work data set if the volumes are
empty.

> From: Neil Duffee 
> To: IBM-MAIN@bama.ua.edu,
> Date: 05/01/2012 03:14 PM
> Subject: Re: Confused on DFSORT SORTWK
> Sent by: IBM Mainframe Discussion List 
>
> In addition to all the other responses you got regarding DynAlloc, I
> wanted to add the nudge that DFSORT only uses the **1st** extent in
> SortWk DD statements.  Thus, SPACE=(CYL,(3335,3335)) is the same as
> SPACE=(CYL,3335) which, in your example, only got you 10*'3335
> cylinders or the equivalent to 10 mod-3s.  So you'd need to have 27
> DD statements to use your entire pool.
>
> >  signature = 6 lines follows  <
> Neil Duffee, Joe Sysprog, uOttawa, Ottawa, Ont, Canada
> telephone:1 613 562 5800 x4585  fax:1 613 562 5161
> mailto:NDuffee of uOttawa.ca http:/ /aix1.uOttawa.ca/ ~nduffee
> "How *do* you plan for something like that?"  Guardian Bob, Reboot
> "For every action, there is an equal and opposite criticism."
> "Systems Programming: Guilty, until proven innocent"  John Norgauer 2004
>
>
> -Original Message-
> From: Mark Pace [mailto:pacemainl...@gmail.com]
> Sent: April 27, 2012 08:43
> Subject: Confused on DFSORT SORTWK
>
> Trying to sort a fairly large RMF data file. 20,000 + cylinders.
> I keep running out of SORTWK space.  I have 9 3390-9 volumes in my
> SMS temporary storage pool. But I just noticed that no matter how
> many SORTWKxx DDs I add it only allocates file on the first 5 temp
> storage volumes.
> Anyone have an idea how I make DFSORT use more volumes?
>
> //SORTWK01 DD DISP=(NEW,DELETE),UNIT=SYSDA,SPACE=(CYL,(3335,3335))
> [snip]
>
> ICE046A E SORT CAPACITY EXCEEDED - RECORD COUNT 2459859
> [snip]
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Confused on DFSORT SORTWK

[David Betten]

��z{S���}�ĝ��xjǺ�*'���O*^��m��Z�w!j�

Re: Confused on DFSORT SORTWK

[David Betten]

Just to clarify, DFSORT supports large format data sets for sortworks thus
allowing them to use more than
64K tracks.  DFSORT's dynamic allocation allocates them as large format.
For JCL allocated sort works
you need to specify DSNTYPE=LARGE on the DD statements and DFSORT will
support that.


>>
My rule of thumb for SORTWK - space is: You need about 1 1/2 to 2 times the
size of SORTIN for your SORTWKs (OK, somebody will probably correct me or
point me to where there's documentation about the real values, but that's
what I have used in the past).
A SORTWK dataset's maximum size is (or used to be when I last messed around
with that a few years ago) 64K tracks (4368 CYL).
So, going out on my old limb, I guesstimate that for 20,000 CYL of input
file you may need between 30,000 and 40,000 CYL of SORTWK space or at least
10 SORTWK DDs. More smaller ones might be better as they allow the disks to
be filled more completely.
If the disks are SMS managed, an overflow - pool might be a good backstop.

Regards,
Ulrich Krueger


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Test

[David Betten]

I think for some reason it's related to when I do a Reply with History.  I
recently went to a newer version of Lotus Notes and suspect something is
now different.  Sorry for the bother to everyone.


David,

The two messages of yours which came through to me garbled had this in the
mail headers:

Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64

The test message had this instead:

Content-Type: text/plain; charset="US-ASCII"

HTH

Peter


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Test

[David Betten]

Sorry about this post.  Trying to determine why my posts are coming out
garbled.

Dave Betten

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Confused on DFSORT SORTWK

[David Betten]

��z{S���}�ĝ��xjǺ�*'���O*^��m��Z�w!j�

Re: Confused on DFSORT SORTWK

[David Betten]

��z{S���}�ĝ��xjǺ�*'���O*^��m��Z�w!j�

Re: z/OS 1.11 DFSORT ICE046A

[David Betten]

There can be many reasons they are now suddenly getting ICE046A.  Can you
send a few sysouts from jobs receiving the ICE046A to the DFSORT Hotline,
dfs...@us.ibm.com ?  Once we determine why they are getting the ICE046A, we
can determine the best approach for addressing it in your environment.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/26/2012
12:49:24 PM:

> From: subscribe ibm-main Jerry Bergman 
> To: IBM-MAIN@bama.ua.edu,
> Date: 03/26/2012 12:54 PM
> Subject: z/OS 1.11 DFSORT ICE046A
> Sent by: IBM Mainframe Discussion List 
>
> Hi,
>
> We recently upgraded from 1.9 to 1.11. It went well, but now several
> of our production jobs are dying with ICE406A - Sort Capacity
> Exceeded.  The JCL has 3 sortwork DDs with an allocation of CYL(1,1)
> or (2,1). Quite frankly I'm surprised they ever worked. When I tell
> the application folks they need to follow the recommendation from
> the doc for ICE046A, namely increase their sortwork size, their
> response is that I need to set some parameter so they don't have to
> change their JCL and if I ask someone will surely know what that
> parameter is and tell me. So I am asking.
>
> Thanks,
> Jerry Bergman
> County of Kern
> bergm...@co.kern.ca.us
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Writing article on telework/telecommuting

[David Betten]

One thing I'll add to that is that if your internet service periodially
drops, it's a real pain if you're connected to a host 3270 session.  For
example, my wife primarally does email and web browsing while working from
home.  So if our internet signal drops for a few minutes and then comes
back, she's not likely to even notice.  However, if I'm scrolling through
code or a hex dump and the sevice drops for just a few seconds, it's a
major headache getting loging back on and hoping my sesson reconnects to
where I was.  Our latest VPN client seems to offer a bit better recovery
from that by maintaining the session but a few years ago it was a major
headache for me.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/04/2012
08:49:57 AM:

> From: Martin Packer 
> To: IBM-MAIN@bama.ua.edu,
> Date: 03/04/2012 10:01 AM
> Subject: Re: Writing article on telework/telecommuting
> Sent by: IBM Mainframe Discussion List 
>
> One experience from teleworking which should appeal to mainframers: By
and
> large 3270 is the least demanding data stream - so TSO / ISPF goes fast
> even on "broadband" as crummy as mine. (It's all the other junk that runs

> really slowly when the wet string dries out.)
>
> Now I may be in a minority but I bet this counts for lots of people.
>
> Anyhow, having telecommuted for more than 10 years I'm looking forward to

> this article: "You are not alone" is a useful thing to hear. :-)
>
> Cheers, Martin
>
> Martin Packer,
> Mainframe Performance Consultant, zChampion
> Worldwide Banking Center of Excellence, IBM
>
> +44-7802-245-584
>
> email: martin_pac...@uk.ibm.com
>
> Twitter / Facebook IDs: MartinPacker
> Blog:
> https://www.ibm.com/developerworks/mydeveloperworks/blogs/MartinPacker
>
>
>
> From:
> Gabe Goldberg 
> To:
> IBM-MAIN@bama.ua.edu,
> Date:
> 03/03/2012 21:43
> Subject:
> Writing article on telework/telecommuting
> Sent by:
> IBM Mainframe Discussion List 
>
>
>
> I'm writing article for Destination z  telework/telecommuting. I think this partitions in two dimensions --
> technology vs. mindset and worker vs. employer.
>
> There's abundant information -- and blather -- about this subject. But
> Destination z is mainframe focused so I'm especially interested in
> relevant System z tips for all four quadrants:
> technology/mindset/worker/employer.
>
> Again, this is a "tips" article so won't include positive/negative
> anecdotes. But they're still welcome -- they can suggest tips, they're
> interesting, and I might write a longer piece on this sometime.
>
> As usual, extra credit for sending to me directly (in addition to list,
> if you're so inclined) so I needn't pluck from digests.
>
> Thanks, as always..
>
> --
> Gabriel Goldberg, Computers and Publishing, Inc.   g...@gabegold.com
> 3401 Silver Maple Place, Falls Church, VA 22042   (703) 204-0433
> LinkedIn: http://www.linkedin.com/in/gabegoldTwitter: GabeG0
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
>
>
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
>
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal sort

[David Betten]

I think you really need to figure out why your JCL sortwork is being
directed to VIO.   Have you tried using a large space allocation.  Usually
ACS routines set a MAXSIZE threshold for directing to a VIO storage group.
If you can stop the SORT work data set from being directed to VIO, then
DFSORT will not try to re-allocate it to SORTDK.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/22/2012
12:17:33 PM:

> [image removed]
>
> Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal
sort
>
> Staffan Tylen
>
> to:
>
> IBM-MAIN
>
> 02/22/2012 12:19 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Sorry Otto, SORTWK01 in the JCL doesn't help. DFSORT then starts to
> allocate SORTDKnn files as real files, not VIO as is the case with
> dynamic allocation.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Avoiding DFSORT dynamic allocatoin, was: Abend S0C4 in an internal sort

[David Betten]

I can't give specific recommendations without knowing more about these
sorts being executed and the environment.  But perhaps I can offer a little
info that will be useful.

The reason the JCL SORTWKxx data sets get reallocated to SORTDKxx is
because VIO=NO is in effect.  This is the installation default we
recommend.  So when the work
data sets are allocated to VIO, DFSORT attempts to re-allocate them to
SORTDKxx data sets using a different unit name in hopes they will be
directed to disk instead of VIO.
Unfortunately, VIO cannot be specified as a run time option so you cannot
change it for just this job.  But perhaps you can code JCL SORTWKs using a
unit that is not directed to
VIO.  I don't know how your ACS routines are set up but usually a VIO
storage class is assigned based on the primary space allocation size.  So
perhaps if you allocated 1 or 2
SORTWK dds with a larger storage size, they'd be assigned to disk and thus
DFSORT would not try to re-allocate them.

When no JCL sortwks are present, then DFSORT's dynamic allocation is used.
Usually, even if the sort is expected to complete in memory, work data sets
are still allocated since
we can't be entirely sure of the file size being passed.  You could at
least reduce the number of work data sets being passed by using an OPTION
statement with DYNALLOC=(,2) which
would reduce the number of work data sets allocated.

There might be some other options we can come up with if we had more info.
Perhaps you can send the entire joblog from an execution of this program to
our DFSORT hot line,
dfs...@us.ibm.com and we can take a look.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/20/2012
12:22:44 PM:

> [image removed]
>
> was: Abend S0C4 in an internal sort
>
> Staffan Tylen
>
> to:
>
> IBM-MAIN
>
> 02/20/2012 12:23 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> I wish to continue this thread as it's related. I'm now running my
> program with internal calls to DFSORT and it works fine except for
> one issue that I have, namely that I get a lot of job log messages
> generated as a result of dynamic allocation of VIO sortwork files.
> DFSORT is invoked 1000's of times by the program (yes, I know but
> there is nothing I can do about that) and each time it creates a
> bunch of job log messages for VIO allocations. For performance I
> want all sorting to be performed in memory, and I don't really care
> at the moment whether it's in hiperspace or dataspace or whatever. I
> successfully used the FILSZ=Ennn parameter until I ran out of space,
> so I increased the value to match the reality and since then I
> cannot get rid of the dynamic allocation messages. I've tried
> DYNALLOC=OFF, HIPRMAX=0, DSPSIZE=0, DSA=0, FILSZ=U0, you name it! in
> all sorts of combinations in an attempt to prevent dynamic
> allocation of work files, all in vain. I've tried region sizes far!
>   too big including REGION=0M. I've also tried preallocating
> SORTWKxx to VIO data sets, but DFSORT then allocates its own
> SORTDKxx data sets instead :(
>
> So how can I successfully sort my records in storage without using
> dynamic allocation with all the rubbish job log messages as a result?
>
> Many thanks,
> Staffan
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Lots of *MASTER* CPU during a big sort

[David Betten]

DFSORT was likely utilizing available central storage for memory object or
Hiperspace sorting.  Normally I have not seen that cause spikes in Master
cpu utilization but perhaps there was a spike in paging that drove the
Master cpu.  If so, there are DFSORT installation defaults that can be set
to limit DFSORT's use of central storage to minimize high paging.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/25/2012
10:08:43 AM:

> [image removed]
>
> Lots of *MASTER* CPU during a big sort
>
> David Andrews
>
> to:
>
> IBM-MAIN
>
> 01/25/2012 10:14 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> I glanced at the SDSF 'DA' frame this morning and noticed one of our
> applications programmers was sorting a large file - something like
> 400,000 tracks.  The system was otherwise lightly loaded and SDSF was
> reporting 2M real storage frames in use by his job.
>
> But what also caught my eye was unusually large CPU consumption by the
> *MASTER* address space: 60-75% continuously while his sort was
> executing.  He cancelled it, and *MASTER* went back to its usual docile
> self.
>
> What is a large DFSort with 2M+ real storage doing to consume that much
> time in *MASTER*?  Is this a characteristic of memory objects?
>
> --
> David Andrews
> A. Duda & Sons, Inc.
> david.andr...@duda.com
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: DFSORT manual humour

[David Betten]

Back when we updated the pubs for V1R10, I thought I got rid of all those
outdated references.  Looks like I missed one.  I'll talk to Frank about
updating that one for our next release.


To your question, you are correct.  If DFSORT's dynamic allocation is being
used, then DFSORT calculates the total work space requirement and spreads
it across the number of work data sets specified in the DYNALOC parameter.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/21/2012
10:30:43 PM:

> [image removed]
>
> DFSORT manual humour
>
> R.S.
>
> to:
>
> IBM-MAIN
>
> 01/21/2012 10:39 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Quote from DFSORT manual:
> ---
> For best performance, specify an emulated 3390-9 device (such as RAMAC)
> or another high-speed IBM disk device as the default, and avoid
> specifying a tape, virtual (VIO), or real 3390-9 devices as the default.
> ---
>
> I have to admit the manual is a little bit obsolete - it is dated on
> 2009. However I'm still under impression of high-speed RAMAC devices.
>
>
>
> BTW, now seriously:
> I can specify number of dynamically allocated work datasets via DYNALOC,
> but the size of those dataset is controlled by DFSORT and user cannot
> change it.
> Is it true?
>
> --
> Radoslaw Skorupka
> Lodz, Poland
>
>
> --
> Tre tej wiadomoci moe zawiera informacje prawnie chronione
> Banku przeznaczone wycznie do uytku subowego adresata. Odbiorc
> moe by jedynie jej adresat z wyczeniem dostpu osób trzecich.
> Jeeli nie jeste adresatem niniejszej wiadomoci lub pracownikiem
> upowanionym do jej przekazania adresatowi, informujemy, e jej
> rozpowszechnianie, kopiowanie, rozprowadzanie lub inne dziaanie o
> podobnym charakterze jest prawnie zabronione i moe by karalne.
> Jeeli otrzymae t wiadomo omykowo, prosimy niezwocznie
> zawiadomi nadawc wysyajc odpowied oraz trwale usun t
> wiadomo wczajc w to wszelkie jej kopie wydrukowane lub zapisanena dysku.
>
> This e-mail may contain legally privileged information of the Bank
> and is intended solely for business use of the addressee. This e-
> mail may only be received by the addressee and may not be disclosed
> to any third parties. If you are not the intended addressee of this
> e-mail or the employee authorised to forward it to the addressee, be
> advised that any dissemination, copying, distribution or any other
> similar activity is legally prohibited and may be punishable. If you
> received this e-mail by mistake please advise the sender immediately
> by using the reply facility in your e-mail software and delete
> permanently this e-mail including any copies of it either printed or
> saved to hard drive.
>
> BRE Bank SA, 00-950 Warszawa, ul. Senatorska 18, tel. +48 (22) 829
> 00 00, fax +48 (22) 829 00 33, www.brebank.pl, e-mail: i...@brebank.pl
> Sd Rejonowy dla m. st. Warszawy XII Wydzia Gospodarczy Krajowego
> Rejestru Sdowego, nr rejestru przedsibiorców KRS 025237, NIP:
> 526-021-50-88.
> Wedug stanu na dzie 01.01.2012 r. kapita zakadowy BRE Banku SA
> (w caoci wpacony) wynosi 168.410.984 zotych.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: Number of SORT WORKs needed

[David Betten]

If they are MOD-27s, you should be able to use more than 4300 cylinders on
each volume.  Just make sure to code DSNTYPE=LARGE on the JCL SORTWK
statements.

For a sort this large, I would also make sure your DFSORT installation
default for DSA is set very large, like DSA=512.  If you run into problems
with this sort, feel free to send a note to our DFSORT hotline,
dfs...@us.ibm.com

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/20/2012
10:15:53 AM:

> [image removed]
>
> Number of SORT WORKs needed
>
> af dc
>
> to:
>
> IBM-MAIN
>
> 01/20/2012 10:21 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hello,
> we have an DB2 unload that needs to be sorted, it has 900.000 CYLs,
> I'm detemining how many sortwkxx must I put on jcl to perform
> this...And according to my calculation using 255 sortwk files with
> 4300 cyls as primary allocation it will not be enough.Storage group
> for temporary files has a lot of free space with mod27 available.
>
> 900.000 * 2 = 1.800.000 cyls  / 4300 = 419 sortwk  !!
>
> Z/Os V1.12
> Can you pls give me an help ?? Any hint is welcome.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: INFO IBM-MAIN

Re: z/OS Tag Sort

[David Betten]

Hello Yifet,
Sorry for the delayed reply but I was on vacation.  I really can't go
into specifics of the algorithm since that is part of our proprietary
internals.  We generally recommend that our customers use MAINSIZE=MAX and
let DFSORT's dynamic storage adjustment calculate the optimal amount of
virtual storage.  I can give you some general ranges of what DFSORT would
calculate as optimal for various file sizes.

File SizeOptimal virtual storage
 32GB  ~ 64MB
 64GB ~100MB
128GB   ~150MB
256GB   ~200MB
512GB   ~256MB
1TB~350MB


These are just approximations of what you'd observe if you executed sorts
of these sizes with no restrictions on region, MAINSIZE=MAX and DSA large
enough not to limit.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/18/2011
07:39:10 AM:

> [image removed]
>
> Re: z/OS Tag Sort
>
> Yifat Oren
>
> to:
>
> IBM-MAIN
>
> 10/18/2011 07:50 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hi David,
>
> You wrote:
>
> > .. That exit is limiting the sort to just 24MB of virtual storage when
the
> optimum amount would be closer to 200MB.
>
> The data size was 360 GB (336m records).
>
> Can you please share the formula you've used to determine the optimum
amount
> is around 200MB?
>
> The DFSORT Tuning Guide (1.12) seems to think 2GB is the upper limit when
it
> comes to recommending minimum virtual storage settings :)
>
> Thanks,
> Yifat Oren
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: z/OS Tag Sort

[David Betten]

ICE162I 0 ICEIEXIT CHANGED ONE OR MORE OPTIONS IN EFFECT

Unfortunately we don't display what options were changed.

Also in the ICE132I Options message I saw IEXIT=Y, TEXIT=Y which indicate
both an installation and a termination exit are in use on this system.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/14/2011
08:35:35 AM:

> [image removed]
>
> Re: z/OS Tag Sort
>
> Elardus Engelbrecht
>
> to:
>
> IBM-MAIN
>
> 10/14/2011 08:37 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> David Betten  wrote:
>
> >The Intermediate Merge is the main cause of the high elapsed time.
> However, in John's case, just increasing DSA is not going to help
> because there is an installation exit (ICEIEXIT) controlling the
> amount of virtual storage DFSORT can allocate.  That exit is
> limiting the sort to just 24MB of virtual storage when the optimum
> amount would be closer to 200MB.
>
> Interesting. Are there any ICE message(s) showing that an exit is
> really in use/called?
>
> Just curious if you don't mind, please.
>
>
> >As for the size of the sort, 336 million records (360GB), I have
> seen much larger sorts than that so I know DFSORT can handle it.
> We've actually executed sorts with record counts in the billions.
>
> Uhhh, I'm feeling SORTa small... :-D
>
> Groete / Greetings
> Elardus Engelbrecht
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: z/OS Tag Sort

[David Betten]

John sent me his complete sysout.  The Intermediate Merge is the main cause
of the high elapsed time.  However, in John's case, just increasing DSA is
not going to help because there is an installation exit (ICEIEXIT)
controlling the amount of virtual storage DFSORT can allocate.  That exit
is limiting the sort to just 24MB of virtual storage when the optimum
amount would be closer to 200MB.


As for the size of the sort, 336 million records (360GB), I have seen much
larger sorts than that so I know DFSORT can handle it.   We've actually
executed sorts with record counts in the billions.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/14/2011
06:59:11 AM:

> [image removed]
>
> Re: z/OS Tag Sort
>
> Elardus Engelbrecht
>
> to:
>
> IBM-MAIN
>
> 10/14/2011 07:23 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> John Blythe Reid wrote:
>
> >ICE247I 0 INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED
>
> Good Eye Catcher. Thanks for stating it here.
>
>
> >ICE054I 0 RECORDS - IN: 336718372, OUT: 336718372
> >ICE134I 0 NUMBER OF BYTES SORTED: 378660163010
> >ICE253I 0 RECORDS SORTED - PROCESSED: 336718372, EXPECTED: 367987636
>
> Oh, that's just a few records... :-D
> No seriously, that is a very serious lot of records. I really doubt
> that I even reached that quantity in ONE sort job in my whole career...
>
>
> >ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 1124, EXPECTED: 1050
> >ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 14030580 , TRACKS
> USED: 14022900
>
> Are you using static or dynamic work datasets? In my humble opinion,
> I would go to static work datasets (SORTWKxx DD), but then this is
> my opinion and preference. Perhaps it is not suitable for you.
>
>
> >ICE090I 0 OUTPUT LRECL = 2100, BLKSIZE = 32760, TYPE = VB   (SDB)
>
> Perhaps you could try out this system determined blocksize of 27998?
>
>
> >ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
> >ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
> >ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
>
> Zeroes only? Where are your ICE093I and ICE156I messages? Can't you
> post them here too, please?
>
>
> I would like to add to Paolo Cacciari's reply (increase DSA):
>
> Could you perhaps increase your job REGION, say to 512M or so?
>
> Also consider adding these options, which helped me to sort out big sort
jobs:
>
> OPTION DYNSPC=512,SIZE=E9,MAINSIZE=MAX
>
> The DFSORT Team may add better recommendations.
>
>
> >There are no sort exits being used.
> >There are no specific limitations on sorting by WLM or any other
process.
>
> Good. I'm glad.
>
>
> Hopes that helps much to SORT out your problem!!! :-D
>
> Groete / Greetings
> Elardus Engelbrecht
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: z/OS Tag Sort

[David Betten]

Send the full joblog/sysout from the sort to dfs...@us.ibm.com and we can
take a look to see if there are any ways to tune the large sort for you.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/13/2011
05:54:04 AM:

> [image removed]
>
> z/OS Tag Sort
>
> John Blythe Reid
>
> to:
>
> IBM-MAIN
>
> 10/13/2011 05:54 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> We are sorting a very large file and it's taking about fourteen hours. I
> remember that there used to be a technique whereby the sort wrote out a
list
> of record addresses and an application program could read the list to
access
> the records in the desired sequence.
>
> Does anyone know if there is still a way of doing this ?
>
> Thanks.
>
> Regards,
> John.
>
>
> --
> John Blythe Reid,
> Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
> Barcelona,
> España.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/03/2011. (returning 10/03/2011)

[David Betten]

I am out of the office until 10/03/2011.

I am out of the office beginning Friday , 9/30 and returning on Monday,
10/03.  My backup is Vicky Vezinaw, vezi...@us.ibm.com


Note: This is an automated response to your message  "Re: OT to my
collegues" sent on 9/29/11 22:59:20.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: elapsed time for DFSORT

[David Betten]

There is not enough information here to determine why the job ran longer.
Please send the complete joblog (not just the DFSORT messages) to
dfs...@us.ibm.com and we can analyze for you.  Also, be sure to send
joblogs from multiple days so we can see the variance in elapsed time.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 09/20/2011
12:38:45 AM:

> [image removed]
>
> elapsed time for DFSORT
>
> Wang Xiaobing
>
> to:
>
> IBM-MAIN
>
> 09/20/2011 12:39 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hi,
>
> Here we have a performance problem on DFSORT.
> The same DFSORT job run on different day have different elapsed
> time, the data processed is almost same.
>
> Form the job log, we find the different is the longer job show
> DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK DATA SET
> SPACE, but our DFSORT job aloways use Hiperspace instead of work
> data set space.
>
> Here is DFSORT log:
>
>  ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0
> ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
>  ICE128I 0 OPTIONS:
> SIZE=57313801,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16
> ,MSGDDN=SYSOUT
>  ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO
> ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA   ,004),ABCODE=MSG
>  ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109
> ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
>  ICE131I 0 OPTIONS:
> TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
>  ICE132I 0 OPTIONS:
> VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE
> ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
>  ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX
> ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
>  ICE235I 0 OPTIONS: NULLOUT=RC0
> ...
>  ICE258I 0 DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK
> DATA SET SPACE
>  ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 33480 , TRACKS USED: 0
>  ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
>  ICE180I 0 HIPERSPACE STORAGE USED = 11628272K BYTES
>  ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
> ...
>
> And, we checked the SMF TYPE30 for these 2 job, it shows the total
> connect time(SMF30TCN) obviously increased..and the  increased
> connect time(SMF30DCT )  is only used for SYSOUT DD..
>
> My question is :
> Q1, why DFSORT COULD NOT DYNAMICALLY ALLOCATE THE OPTIMAL WORK DATA
> SET SPACE ? and how to correct it ? we do not change the JOB.
>
> Q2, In this case, we use  Hiperspace instead of work data set space,
> ICE258I will impace the total connect time ? and cause the elapsed
> become longer ?
>
> Q3, What's the possible root reason increased the  total connect
> time(SMF30TCN) ?
>
> Welcome any comments...TIA.
>
> WXB.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SLR columns size D_MIGR_SP mbyte gbyte

[David Betten]

Wow, this really takes me back.  I believe what you have to do is edit the
ASM source for the table and recompile/linkedit it.  Based on the column
names it looks like you are using th eHSM_MIGR_SUM summary table (module
DREG0E3D) which is built off of the HSM_MIGRATE log table (module
DREG0E2D).  The SLR product did make available the assembler source so
check your SLR libraries.  If can find the source for DREG0E2D and
DREG0E3D, the definition for these columns probably contains something like
EDIT=F(8).  You probably want to change that to EDIT=F(12) and then
recompile.  You may have to rerun your collect with the updated tables
prior to rerunning your reports.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 09/08/2011
04:12:35 AM:

> [image removed]
>
> SLR columns size D_MIGR_SP mbyte gbyte
>
> MONTERO ROMERO, ENRIQUE ELOI
>
> to:
>
> IBM-MAIN
>
> 09/08/2011 04:14 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hi team,
>
> This time is related to the OLD product (SLR).
>
> I am looking for the guides (reference or commands)
>
> We have this instruction :
>
> SELECT COLUMNS(YEAR MONTH DAY ALLOC_SP D_MIGR_SP T_MIGR_SP T_BCK_SP  +
> NUMDATASET NUM_DASD NUM_TAPE NUM_BKUP)
>
> The D_MIGR_SP and T_MIGR_SP results appears with "", this
> means the value cannot fit in the report column.
>
> The solution: (not to upgrade to a newer version), could be to
> divide into 1024 or to increase the columns size, but we have no
> idea on how to do it.
>
> Some clue or guide?
>
> Best regards,
>
> Enrque Eloi Montero
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/21/2001. (returning 08/22/2011)

[David Betten]

I am out of the office until 08/22/2011.

I am on vacation beginning Wednesday, 8/17 and returning on Monday, 8/22.
My backup is Vicky Vezinaw, vezi...@us.ibm.com


Note: This is an automated response to your message  "Re: Re : A question
on Menu's" sent on 8/16/11 20:56:39.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office until 10/21/2001. (returning 06/13/2011)

[David Betten]

I am out of the office until 06/13/2011.

I am on vacation Friday, 6/10, returning on Monday, 6/13.  My backup is
Vicky Vezinaw, vezi...@us.ibm.com


Note: This is an automated response to your message  "jes2 cold Start" sent
on 6/10/11 3:57:22.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT: DYNAUTO=NO?

[David Betten]

I can't get into specifics of our internal algorithms but I can give you
some explanations.

As for why you are still seeing 6 work data sets allocated for sorts
completed "in core".  You are probably seeing message ICE080I IN MAIN
STORAGE SORT.  DFSORT tends to be a bit cautious about bypassing allocation
of work data sets.  We look at the main storage allocated as compared to
the file size along with other factors like whether an E15 is present and
then only bypass dynamic allocation if we are extremely confident the sort
can complete entirely in main storage.  Otherwise, we allocate the default
number of work data sets which in your case is 6 due to your DYNALOC
default of (WOKDA,6).

As for those cases where DFSORT will force dynamic allocation even if you
have DYNAUTO=OFF.  Those cases involve DFSORT's use of data space,
Hiperspace or memory object where we believe there may be enough storage to
complete the sort without work data sets but we're not overly confident.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/07/2011
11:28:57 AM:

> [image removed]
>
> DFSORT: DYNAUTO=NO?
>
> Chase, John
>
> to:
>
> IBM-MAIN
>
> 03/07/2011 11:29 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hello, DFSORT Team (in particular),
>
>
>
> We're on z/OS 1.11, and one of our programmers was "cleaning up" some
> JCL by removing explicit SORTWKnn DD statements, among other things.  In
> all job streams he's tested so far, all the sorts were accomplished "in
> core"; yet the JES message log shows allocation and deletion of SORTWK01
> thru SORTWK06 in every SORT step.
>
>
>
> What I think the relevant DFSORT installation options we have in place
> are these:
>
>
>
> DYNALOC=(WORKDA,6)
>
> DYNAUTO=YES
>
> DYNSPC=256
>
>
>
> There is also this note in the text for DYNAUTO:
>
> "Even if you specify DYNAUTO=NO, DFSORT can enable automatic dynamic
> allocation in some cases. Therefore, you should always use DYNALOC to
> specify the defaults for the device and number of work data sets for use
> with dynamic allocation. "
>
> What are ("some of") the cases when DFSORT "can enable dynamic
> allocation" when DYNAUTO=NO?
>
>
>
> TIA,
>
>
>
> -jc-
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SMF data for DFSORT

[David Betten]

DFSORT reports cpu time in field ICECPUT.  This is basically the step TCB
time accumulated between the sort start time and the sort end time.  So
that time is a subset of the TCB time reported in type 30 records.  Also
keep in mind that it also means if there are things like user written E15
and E35 exits being used, it's going to include the time spent in those
exits so it's not pure DFSORT only cpu time.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/01/2010
09:51:31 AM:

> [image removed]
>
> SMF data for DFSORT
>
> Michael Hall
>
> to:
>
> IBM-MAIN
>
> 12/01/2010 09:52 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Is there additional information about CPU time for DFSORT in the SMF Type
16
> record that is not in the Type 30 step record. In other words, are there
any
> circumstances where CPU time data is written to the Type 16 records and
not
> to the Type 30 records? Do you see step information for DFSORT CPU time
in
> Type 30 records when DFSORT is indirectly invoked from another program?
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT chooses not to use Hipersorting for large SORTIN

[David Betten]

DFSORT evaluates the available resources and the characteristics of each
sort to determine whether or
not it would be beneficial to exploit Hiperspace, memory object or
dataspace.  Even though your system
is not paging, it's possible that DFSORT determined there was not enough
central strorage available to
be of benefit.  If you would like more detailed analysis, send a note with
your joblog to the DFSORT hotline,
dfs...@us.ibm.com and we can look more closely at it.  It would also be
helpful if you were able to rerun the
sort with //SORTDIAG DD DUMMY added to the JCL.  This causes DFSORT to
write additional diagnostic
message to sysout.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/06/2010
01:19:53 PM:

> [image removed]
>
> DFSORT chooses not to use Hipersorting for large SORTIN
>
> Yifat Oren
>
> to:
>
> IBM-MAIN
>
> 10/06/2010 01:14 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hello All,
>
> I have a sort of about 4GB, 15M records of variable length.
>
> For some reason DFSORT chooses not to use dataspace/hiperspace or memory
> objects for this sort for no obvious reason;
>
> Parameters are all set corrects (EXPMAX, RES, OLD and HIPRMAX and so on).
> The system is not paging at all.
>
> Only thing not tuned for this SORT (as far as I can see) is that it does
not
> know the AVGRLEN in advance, and wrongly calculated the
> number of expected records in the SORTIN (it expects 350k instead of 15m,
> based on the SORTIN LRECL).
>
> So, DFSORT volunteered not to use Hipersorting.
>
> Looking for answers I found this excerpt from DFSORT Tuning Guide on
> Hipersorting:
>
> When Hipersorting cannot be used, DFSORT uses disk work data sets to
store
> its intermediate data, which is referred to as disk-only
> mode. Note that Hiperspace-only mode usually provides the best
performance
> when compared to Hiperspace-mixed and disk-only
> modes. However, this is not always true for Hiperspace-mixed mode when
> compared to disk-only mode. Due to the additional
> Hiperspace overhead, the use of disk-only rather than Hiperspace-mixed
mode
> can at times be more advantageous in terms of
> performance, and therefore DFSORT may choose not to use Hipersorting.
>
>
>  So, what are those "times"; what made DFSORT use disk-only mode? The
file
> size (4GB)? The variable-length? The wrong records number?
>
> Any ideas would be appriciated,
> Yifat
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 10/06/2010)

[David Betten]

I am out of the office until 10/06/2010.

I am on vacation beginning Wednesday, September 29th, returning on
Wednesday, October 6th.  If you need immediate assistance contact my backup
Vicky Vezinaw (vezi...@us.ibm.com) or my manager, Marc Casad
(mdca...@us.ibm.com)


Note: This is an automated response to your message  "Re: IPLTEXT and
NUCLEUS dates" sent on 9/29/10 0:50:34.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SORT question

[David Betten]

I think the real question was how to tell sort to stop reading anymore
input records when it reaches the first record that's greater than the
selection criteria instead of reading through the rest of the file but not
selecting any of those records.  I'm not sure we have a way to do that but
I'm sure Frank will weigh in shortly.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 09/28/2010
10:41:46 AM:

> [image removed]
>
> Re: SORT question
>
> Farley, Peter x23353
>
> to:
>
> IBM-MAIN
>
> 09/28/2010 10:42 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
> > Behalf Of Richard Pinion
> > Sent: Tuesday, September 28, 2010 10:31 AM
> > To: IBM-MAIN@bama.ua.edu
> > Subject: SORT question
> >
> > I have a large file that is sorted in ascending order by a date field,
> > MMDD.  I want to run a sort fields=copy with selection criteria
> > against this file.  One of the selection criteria is the date field.  I
> > would like the sort to terminate once the date field in the file is
> > greater than my selection date.  Is this possible?
>
> OPTION COPY
> OMIT COND=(datepos,datelen,CH,GT,'MMDD')
>
> where MMDD is your selected date.  If date is not in character
> format, change CH and date literal to match actual format.
>
> You don't need the SORT keyword since this is just a copy.
>
> HTH
>
> Peter
> --
>
> This message and any attachments are intended only for the use of
> the addressee and
> may contain information that is privileged and confidential. If the
> reader of the
> message is not the intended recipient or an authorized representative of
the
> intended recipient, you are hereby notified that any dissemination of
this
> communication is strictly prohibited. If you have received this
> communication in
> error, please notify us immediately by e-mail and delete the message and
any
> attachments from your system.
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Query on IMS - OSAM

[David Betten]

I believe the discrepancy you are seeing is due to the 8KB block size not
allowing you to fully utilize each track.  You are probably only getting 6
blocks per track or 49512 bytes since 7 blocks would put you over the track
capacity.  Now if you muliply your used tracks of 135765 by 49512 that
comes pretty close to your 6.15GB calculation.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 09/24/2010
02:14:14 PM:

> [image removed]
>
> Query on IMS - OSAM
>
> Veena, Sridhar
>
> to:
>
> IBM-MAIN
>
> 09/24/2010 02:24 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List
>
> Hi,
>
>
>
> I need your help in addressing an OSAM space allocation concern. One of
> our OSAM database dataset is having allocated space = 150120 tracks and
> used space = 135765 tracks. The allocation is on 3390 device. If we take
> one track = 56KB for 3390 device type and do the math, then it looks
> like our database dataset is already at the allocated space = 8GB (
> 150120 * 56KB). This means we cannot allocate more space to the dataset.
> IMS cannot address OSAMs beyond 8GB.
>
>
>
> But on verifying the re-org output space stats, the DB dataset is
> containing 806868 blocks of 8KB each. This suggests the database dataset
> size to be at 6.15GB (806868 * 8KB). We are confused on which size to
> consider and whether we can add more space to our OSAM dataset which is
> at 150120 tracks now (on 3390 device).
>
>
>
> Thanks & Rgds
>
> Sridhar K Veena
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 08/17/2010)

[David Betten]

I am out of the office until 08/17/2010.

I am on vacation August 11-16, returning on Tuesday, August 17.  If you
need immediate assistance contact my backup Vicky Vezinaw
(vezi...@us.ibm.com) or my manager, Marc Casad  (mdca...@us.ibm.com)


Note: This is an automated response to your message  "Re: auditor request
question" sent on 8/11/10 11:59:18.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Dynamic sortwork

[David Betten]

No DFSORT does not do anything like that.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/07/2010
02:20:56 AM:

> [image removed]
>
> Re: Dynamic sortwork
>
> Elardus Engelbrecht
>
> to:
>
> IBM-MAIN
>
> 07/07/2010 02:21 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> David Betten wrote:
>
> >Yes it is correct.  We try to allocate all the required space usingn-1
work
> data sets and we keep that 1 additional to use if the work space
requirement
> is larger than expected.
>
> Thanks for kindly helping Radoslaw. I also learned something new today.
>
> A question, if you don't mind please, do DFSORT (and perhaps ICETOOL)
also
> use that 1 additional dataset in case there is an ABEND and data needed
to
> be recovered or moved out of the way?
>
> Again thanks
>
> Groete / Greetings
> Elardus Engelbrecht
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Dynamic sortwork

[David Betten]

Yes it is correct.  We try to allocate all the required space using n-1
work data sets and we keep that 1 additional to use if the work space
requirement is larger than expected.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/06/2010
11:21:25 AM:

> [image removed]
>
> Dynamic sortwork
>
> R.S.
>
> to:.
>
> IBM-MAIN
>
> 07/06/2010 11:22 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I observed that DFSORT does use n-1 dynamic sortwk files.
> For n=4 only 3 files have non-zero space. Fourth is empty (only VTOC
> entry). It's unrelated to number of files or space constraints.
> For the same data to be sorted one can increase n and still gets approx.
> the same total number of tracks, and one of the files is empty.
>
> Q1: Is the above correct?
> Q2: What is the purpose of the empty file?
>
> --
> Radoslaw Skorupka
> Lodz, Poland
>
>
> --
> BRE Bank SA
> ul. Senatorska 18
> 00-950 Warszawa
> www.brebank.pl
>
> Sąd Rejonowy dla m. st. Warszawy
> XII Wydział Gospodarczy Krajowego Rejestru Sądowego,
> nr rejestru przedsiębiorców KRS 025237
> NIP: 526-021-50-88
> Według stanu na dzień 01.01.2009 r. kapitał zakładowy BRE Banku SA
> (w całości wpłacony) wynosi 118.763.528 złotych. W związku z
> realizacją warunkowego podwyższenia kapitału zakładowego, na
> podstawie uchwały XXI WZ z dnia 16 marca 2008r., oraz uchwały XVI
> NWZ z dnia 27 października 2008r., może ulec podwyższeniu do kwoty
> 123.763.528 zł. Akcje w podwyższonym kapitale zakładowym BRE Banku
> SA będą w całości opłacone.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

AUTO: David Betten/Gaithersburg/IBM is out of the office (returning 07/12/2010)

[David Betten]

I am out of the office until 07/12/2010.

I am on vacation July 5-9, returning on Monday, July 12.  If you need
immediate assistance contact my backup Vicky Vezinaw (vezi...@us.ibm.com)
or my manager, Marc Casad  (mdca...@us.ibm.com)


Note: This is an automated response to your message  "Re: GML (was Unix
systems and Serialization mechanism)". sent on 7/2/10 22:09:06.

This is the only notification you will receive while this person is away.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: supported releases of DFSORT

[David Betten]

Correct.

> >
> > Thanks, so V1.5 is the oldest supported release, correct.
>
> Jim McAlpine
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: supported releases of DFSORT

[David Betten]

They coincide with the z/OS EOS dates.  DFSORT V1R5 is the DFSORT release
for z/OS 1.5 thru 1.9.  So the EOS for z/OS 1.9 is the EOS for DFSORT V1R5.
I don't know what that date is off the top of my head.  I can look it up if
you need it.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/02/2010
09:12:02 AM:

> [image removed]
>
> supported releases of DFSORT
>
> Jim McAlpine
>
> to:
>
> IBM-MAIN
>
> 07/02/2010 09:12 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I can't find DFSORT mentioned on the IBM Software Support Lifecycle site
and
> the DFSORT site doesn't seem to mention EOS dates.  Can someone point me
to
> EOS dates for DFSORT.
>
> Jim McAlpine
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT and SORTWK space

[David Betten]

Your assumptions are correct.  You can increase the total amount of work
space possible by increasing the number of work data sets that DFSORT
dynamically allocates.   This is by using the n value in the DYNALOC
installation default or the DYNALLOC run time options.  For example

OPTION DYNALLOC=(SYSDA,8)



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 06/20/2010
01:30:38 AM:

> [image removed]
>
> DFSORT and SORTWK space
>
> R.S.
>
> to:.
>
> IBM-MAIN
>
> 06/20/2010 01:31 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> If you omit SORTWKnn DDnames then - assumed default values used - DFSORT
> will use 3 datasets, in LARGE format. Every sort work dataset is single
> volume, so the limit for default sort work space depends on size (and
> free space) of work volumes.
> Q: Am I right with the above?
>
> For example, I have several mod-54 work volumes, so my sort work space
> limit is approximately 3x65500 cylinders.
>
> Further question: What if volumes are framgmented? LARGE PS has limit of
> 16 extents, so the volume fragmentation could lead to smaller maximum
> sizes - is it true?
>
>
> Regards
> --
> Radoslaw Skorupka
> Lodz, Poland
>
>
> --
> BRE Bank SA
> ul. Senatorska 18
> 00-950 Warszawa
> www.brebank.pl
>
> Sąd Rejonowy dla m. st. Warszawy
> XII Wydział Gospodarczy Krajowego Rejestru Sądowego,
> nr rejestru przedsiębiorców KRS 025237
> NIP: 526-021-50-88
> Według stanu na dzień 01.01.2009 r. kapitał zakładowy BRE Banku SA
> (w całości wpłacony) wynosi 118.763.528 złotych. W związku z
> realizacją warunkowego podwyższenia kapitału zakładowego, na
> podstawie uchwały XXI WZ z dnia 16 marca 2008r., oraz uchwały XVI
> NWZ z dnia 27 października 2008r., może ulec podwyższeniu do kwoty
> 123.763.528 zł. Akcje w podwyższonym kapitale zakładowym BRE Banku
> SA będą w całości opłacone.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: icetool help with short fields

[David Betten]

Frank will correct me if I'm mistaken but I believe you might need the
VLSCMP option.  This will cause DFSORT to pad "short" variable
length INCLUDE/OMIT compare fields with binary zeroes.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/01/2010
12:07:20 PM:

> [image removed]
>
> Re: icetool help with short fields
>
> Bri P
>
> to:
>
> IBM-MAIN
>
> 03/01/2010 12:08 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> Well the message is telling you that you're expecting a field at
> least 28 bytes long (because of the 19,10,CH statement) but you've
> only read in a record of 26 bytes.
>
> That does seems a bit strange. I mean, that date field is surely
> either going to be there in its entirety or not at all, not just
> have two bytes missing?
>
> I can't see anything wrong with the SORT or ICETOOL statements.
>
> Is your source IRRDBU00 extract ok? (browse it - it's nearly all
> readable text data)
>
> Maybe also write the TMP0001 extract to a non-temporary file and
> examine that too.
>
> Brian
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
> Behalf Of larry macioce
> Sent: 01 March 2010 16:14
> To: IBM-MAIN@bama.ua.edu
> Subject: icetool help with short fields
>
> I was graciously given some jcl to do a sort of an unloaded racf db.
> The problem is I am getting:
> ICE218A 1 26 BYTE VARIABLE RECORD IS SHORTER THAN 28 BYTE MINIMUM FOR
>FIELDS
> the jcl is:
>
> //TOOLIN DD *
>  SORTFROM(INDD) TO(TEMP0001) USING(ULDF)
>  DISPLAY FROM(TEMP0001) LIST(PRINT) -
>  PAGE -
>  TITLE('LIST OF IDS') -
>  DATE(4MD/) -
>  TIME(24:) -
>  BLANK -
>  ON(10,8,CH) HEADER('USER ID') -
>  ON(79,20,CH) HEADER('USER NAME') -
>  ON(100,8,CH) HEADER('DEFAULT GROUP') -
>  ON(19,10,CH) HEADER('CREATED')
> //INDD DD DISP=SHR,DSN=SYS1.RACF.RPT
> //TEMP0001 DD DISP=(NEW,DELETE,DELETE),SPACE=(CYL,(20,5,0)),
> //  UNIT=SYSDA
> //ULDFCNTL   DD *
>  SORT FIELDS=(10,8,CH,A)
>  INCLUDE COND=(5,4,CH,EQ,C'0200',AND,
>19,10,CH,GE,C'2008-07-01',AND,
>19,10,CH,LE,C'2009-06-30')  OPTION VLSHRT
>
> As you can see I was given the vlshrt (didnt know what it was, looked it
up)
> option so now I am confused.
> Thanks
> mace
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SORTWK files

[David Betten]

Larry,
  Sounds good.  Also, don't forget about the DFSORT hotline
(dfs...@us.ibm.com) if you have questions or need help debugging a problem
as you develop your solution.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/15/2010
09:28:44 AM:

> [image removed]
>
> Re: SORTWK files
>
> Larry Crilley
>
> to:
>
> IBM-MAIN
>
> 02/15/2010 09:33 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I would not read a single record and pass to each of the sorts.  I will
hope
> to fill multiple "buffers" with many records.  For example, why not get 5
> buffers (or more) and each buffer is 1 Meg (or larger) in size.  Stash as
> many records as possible in each buffer and have each sort pull the
records
> from these buffer areas.  These buffer areas could be above the bar.
I'll
> have to do a little buffer management and know when each sort is done
with
> each buffer, but all in all, not too difficult.
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
Behalf
> Of David Betten
> Sent: Friday, February 12, 2010 8:48 PM
> To: IBM-MAIN@bama.ua.edu
> Subject: Re: SORTWK files
>
> That's a very good point.  You need to take into account how much virtual
> storage is available when running parallel
> sorts in the same address space.  You could limit each sort by passing a
> DSA value to limit DFSORT's dynamic storage
> adjustment from allocating more storage than is available.  This is also
> another good reason to run with as few work
> data sets as possible since they each require virtual storage for control
> blocks, buffers, etc.
>
>
> I'm also starting to wonder just how much elapsed time would be saved by
> using this strategy of having a program
> read the input and then pass to multiple sorts.  With today's world of
PAV
> and high speed devices/channels, is this
> really going to be much faster than having each read the input?  I guess
> testing will bare that out.
>
> This strategy is similar to one we used to see employed with BatchPipes.
> You'd have one reader job which then
> passed the records to multiple concurrent jobs through pipes.  What I
> tended to find, is that you end up serializing the reads
> and the overall result is increased elapsed time.  Basically they are all
> going to read at the pace of the slowest sort.  For
> example you pass records to 4 concurrent sorts.  SORT1 will get record 1
> and then wait until that's been passed to SORT2, SORT3,
> SORT4 before it gets record 2.  Then it waits again for record 3 and so
on
> and so on.  These parallel sorts are probably going to be
> doing a lot of waiting for input.  You can probably minimize this with
some
> sort of large buffer area and wait/post logic but my guess is
> that will be some complex code.
>
>
>
>
> IBM Mainframe Discussion List  wrote on 02/12/2010
> 06:37:47 PM:
>
> > [image removed]
> >
> > Re: SORTWK files
> >
> > Blaicher, Chris
> >
> > to:
> >
> > IBM-MAIN,
> >
> > 02/12/2010 06:38 PM
> >
> > Sent by:
> >
> > IBM Mainframe Discussion List 
> >
> > Please respond to IBM Mainframe Discussion List.
> >
> > I just noticed the following in one of your postings.
> >
> > "If I want to subtask multiple sorts (same input file),"
> >
> > If you are going to run multiple sorts concurrently in the same
> > address space, things can get interesting.  NOTE: You may want to
> > confirm what I say with your sort vendor.
> >
> > Sorts like resources, mainly memory.  They also think they are the
> > only thing running in the address space, or that whatever else is
> > running will not get more memory once the sort has started.  This
> > means that starting multiple sorts can have a very negative impact
> > on the sorts started first.
> >
> > Running multiple sorts will often times get you an 878 type abend,
> > especially if your REGION is not REGION=0M.
> >
> > If all the sorts are similar in size and characteristics, then maybe
> > a simple thing as passing the proper parameters in SORTPARM may do
> > the trick.  If you have sorts of different sizes and different
> > characteristics and run them all with a common set of over-rides,
> > then some are getting more than they need and others are probably
> > getting less.
> >
> > This is not as simple as it

Re: SORTWK files

[David Betten]

That's a very good point.  You need to take into account how much virtual
storage is available when running parallel
sorts in the same address space.  You could limit each sort by passing a
DSA value to limit DFSORT's dynamic storage
adjustment from allocating more storage than is available.  This is also
another good reason to run with as few work
data sets as possible since they each require virtual storage for control
blocks, buffers, etc.


I'm also starting to wonder just how much elapsed time would be saved by
using this strategy of having a program
read the input and then pass to multiple sorts.  With today's world of PAV
and high speed devices/channels, is this
really going to be much faster than having each read the input?  I guess
testing will bare that out.

This strategy is similar to one we used to see employed with BatchPipes.
You'd have one reader job which then
passed the records to multiple concurrent jobs through pipes.  What I
tended to find, is that you end up serializing the reads
and the overall result is increased elapsed time.  Basically they are all
going to read at the pace of the slowest sort.  For
example you pass records to 4 concurrent sorts.  SORT1 will get record 1
and then wait until that's been passed to SORT2, SORT3,
SORT4 before it gets record 2.  Then it waits again for record 3 and so on
and so on.  These parallel sorts are probably going to be
doing a lot of waiting for input.  You can probably minimize this with some
sort of large buffer area and wait/post logic but my guess is
that will be some complex code.




IBM Mainframe Discussion List  wrote on 02/12/2010
06:37:47 PM:

> [image removed]
>
> Re: SORTWK files
>
> Blaicher, Chris
>
> to:
>
> IBM-MAIN,
>
> 02/12/2010 06:38 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I just noticed the following in one of your postings.
>
> "If I want to subtask multiple sorts (same input file),"
>
> If you are going to run multiple sorts concurrently in the same
> address space, things can get interesting.  NOTE: You may want to
> confirm what I say with your sort vendor.
>
> Sorts like resources, mainly memory.  They also think they are the
> only thing running in the address space, or that whatever else is
> running will not get more memory once the sort has started.  This
> means that starting multiple sorts can have a very negative impact
> on the sorts started first.
>
> Running multiple sorts will often times get you an 878 type abend,
> especially if your REGION is not REGION=0M.
>
> If all the sorts are similar in size and characteristics, then maybe
> a simple thing as passing the proper parameters in SORTPARM may do
> the trick.  If you have sorts of different sizes and different
> characteristics and run them all with a common set of over-rides,
> then some are getting more than they need and others are probably
> getting less.
>
> This is not as simple as it sounds.  To put it in perspective, I
> have 25 years of experience in trying to get it right.  Of course, I
> am trying to balance anywhere from 1 to 48 concurrent sorts of
> different types and sizes.
>
> You'll get it to run, but getting it to run the best it can is not easy.

>
> Chris Blaicher
> Phone: 512-340-6154
> Mobile: 512-627-3803
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SORTWK files

[David Betten]

> Many moons ago (maybe DFSORT 1.7) the recommended total SORTWK space was
1.7
> times the input file size. I've noted how using Hiperspace and Dataspace
> reduces the sortwk space used, and I'm wondering if your 10%
recommendation
> is based on Sort being able to use an optimal amount of memory.

Actually our recommendation has always been 1.3 times the file size, not
1.7. But
as I've been looking at more and more customer data, I'm finding that 1.1
is usually
sufficient.  And given the size of today's sorts, 1.3 can cause quite a bit
of
additional space to be allocated.

>
> If so, would you recommend a larger value for SORTWK space on memory
> constrained system, or where customers have hobbled sort's memory in the
> ways that are often discussed on this list? In other words, is 110% good
> enough if sort chooses the smallest dynamic memory options.
>

The 1.1 times the file size recommendation assumes you are not going to
use>
any central storage for sorting in memory.  So it's the worst case.  This
is one
of the reasons we recommend using DFSORT's dynamic allocation since it
takes
into account the expected central storage exploitation and adjusts the disk
work space allocation accordingly.



> Some questions on your SORTWK performance tests: Does CFW on/off affect
the
> impact when many SORTWK datasets are allocated (I'm assuming you mean 16
or
> more), and does the difference between 4 and many SORTWK change in any
way
> when there is an intermediate merge?

Four is really just a good starting point.  I couldn't make a blanket rule
of thumb recommendation to use x if CFW is on and y if CFW is off.  There
are so many other factors like the number of controllers, cache sizes,
I/O contention, etc.


>
> And a query: is the recommendation of 4 SORTWK good up to a particular
size.
> I'm interested in that point because four SORTWK would be limited to
about
> 17K CYLS total.

Again, 4 is just a good starting point.  I think the size of the sort and
the size and
number of volumes in the work pool would really factor in how many work
data sets
you want/need to allocate.  Also, DFSORT supports large format data sets
for work data sets
so we can actually use much more than 17K cylinders with 4 work data sets.
DFSORT's dynamic
allocation automatically allocates the work data sets as large format.




>
> Ron
>
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
> Behalf Of
> > David Betten
> > Sent: Friday, February 12, 2010 6:39 AM
> > To: IBM-MAIN@bama.ua.edu
> > Subject: Re: [IBM-MAIN] SORTWK files
> >
> > For DFSORT, we generally recommend 4 work data sets.  I have done
testing
> > with varied numbers of work data sets and not seen much variance in
> > performance until you start allocating very large numbers of work data
> sets
> > for small sorts.Also, I would recommend that you allocate total
space
> > of about 10% more than the file size.  So for your 100 cylinder data
set,
> > you'd want to allocate 4 work data sets of about 28 cylinders each.
> >
> > Also, we generally don't recommend allocating your own work data sets
and
> > instead let DFSORT's dynamic allocation handle it.  DFSORT can take
into
> > account how much central storage is going to be used and adjust the
work
> > data set space allocations accordingly.
> >
> >
> > Have a nice day,
> > Dave Betten
> > DFSORT Development, Performance Lead
> > IBM Corporation
> > email:  bet...@us.ibm.com
> > DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
> >
> > IBM Mainframe Discussion List  wrote on
02/12/2010
> > 09:13:53 AM:
> >
> > > [image removed]
> > >
> > > SORTWK files
> > >
> > > Larry Crilley
> > >
> > > to:
> > >
> > > IBM-MAIN
> > >
> > > 02/12/2010 09:15 AM
> > >
> > > Sent by:
> > >
> > > IBM Mainframe Discussion List 
> > >
> > > Please respond to IBM Mainframe Discussion List.
> > >
> > > I want to dynamically allocate my own sort work files.  If I know how
> > large
> > > my input file is (I can read the catalog), then what would everyone
> > > recommend for the allocation of the sort work files?  How many
datasets
> > are
> > > optimal?  If my input file is 100 cylinders, for example, would you
> > allocate
> > > 3 sort work files CYL(35,0), 4 sort work files CYL(25,0), 8 sort work
> > > files.?
> > >
> > >
> > >
> > >
> > >
> > > Larry Crilley
> > >
> 

Re: SORTWK files

[David Betten]

>
> DFSORT tells me: "Generally, DFSORT can automatically determine the input
> file size. However, in a few cases, such as when an E15 supplies all of
the
> input records,

Actually it's when E15 supplies all of the input records AND no file size
information is provided
to DFSORT.  If you pass a file size and record length information, then
DFSORT will have the information
it needs to calculate the correct amount of work space as well other
resources like main storage and such.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SORTWK files

[David Betten]

For DFSORT, we generally recommend 4 work data sets.  I have done testing
with varied numbers of work data sets and not seen much variance in
performance until you start allocating very large numbers of work data sets
for small sorts.Also, I would recommend that you allocate total space
of about 10% more than the file size.  So for your 100 cylinder data set,
you'd want to allocate 4 work data sets of about 28 cylinders each.

Also, we generally don't recommend allocating your own work data sets and
instead let DFSORT's dynamic allocation handle it.  DFSORT can take into
account how much central storage is going to be used and adjust the work
data set space allocations accordingly.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/12/2010
09:13:53 AM:

> [image removed]
>
> SORTWK files
>
> Larry Crilley
>
> to:
>
> IBM-MAIN
>
> 02/12/2010 09:15 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I want to dynamically allocate my own sort work files.  If I know how
large
> my input file is (I can read the catalog), then what would everyone
> recommend for the allocation of the sort work files?  How many datasets
are
> optimal?  If my input file is 100 cylinders, for example, would you
allocate
> 3 sort work files CYL(35,0), 4 sort work files CYL(25,0), 8 sort work
> files.?
>
>
>
>
>
> Larry Crilley
>
> Dino-Software Corporation
>
> 800.480.DINO
>
> 412.366.3566
>
>
>
 ware.com> www.dino-software.com
>
>
>
>  http://www.dino-software.com/trex_factsheet.php>
T-REX -
> Superior ICF catalog mgmt with full Tape support and HSM Auditing
>  http://www.dino-software.com/reorgadon_factsheet.php>
> REORGadon - First ever online REORG for HSM
>  http://www.dino-software.com/teradon_factsheet.php>
> TERADON - First ever REPRO MERGECAT While-OPEN
>  http://www.dino-software.com> XTINCT - Secure
DASD/TAPE
> data eradication
>  http://www.dino-software.com/rtd_factsheet.php> RTD -
> DASD Real Time Defrag
>  http://www.dino-software.com/dal_details.php> DAL -
DINO
> healthcheck Analysis service for Legato
>  http://www.dino-software.com/sentinel_factsheet.php>
> SENTINEL - Real-time FTP Management. All Secure, all the time.
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: why compression costs additional I/O?

[David Betten]

> generally all of it probably mean that using DFSORT for compressed
datasets is
> not good idea.

I'm not sure I would agree with a general statement such as that.

First.  There is a cpu overhead associated with compression and it effects
ALL applications, not just sort.  The overhead
is generally higher for write than it is for read.  In some cases, that
overhead can be offset by reduced data transfer but that
depends on how well the data compresses.  You also need to look at how the
data set is used.  If it's written once but read
many times, then you may get enough benefit on all those reads to warrant
the negative impact on the write.

Second, higher EXCPs does not necessarily mean higher I/Os.  For BSAM,
buffers are used to store the blocks which are
then chained together in single I/Os.  So the increase in I/Os is likely
much smaller than the increase in EXCPs.

Thirdly, you may want to consider multiple stripes so that data is
transferred in parallel.  This won't reduce the I/Os but
it would allow multiple I/Os to be done in parallel and reduce elapsed
time.


I've never really considered compression as a means of improving
performance.  I've heard all the arguments about less
data being transferred but in all my years of batch tuning I never really
saw that great an impact to offset the cpu cost.  To me,
compression is great for avoiding out of space conditions and managing very
large files.  When performance is the sole
concern, I've always recommended extended format with multiple stripes but
not compressed.   Of course that requires that.
you have the disk space available to support storing the large data sets!


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/27/2010
10:23:22 AM:

> [image removed]
>
> Re: why compression costs additional I/O?
>
> Pawel Leszczynski
>
> to:
>
> IBM-MAIN
>
> 01/27/2010 10:26 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> Hi Yifat,
>
> Thanks for answer - you are right! - I 've checked in joblog:
>
> for compressed output:
>
>  0 SORTOUT  : BSAM USED
>
> but for non-compressed output:
>
> SORTOUT  : EXCP USED
>
> generally all of it probably mean that using DFSORT for compressed
datasets is
> not good idea.
>
> Regards,
> Pawel
>
>
>
>
>
> On Wed, 27 Jan 2010 15:55:24 +0200, Yifat Oren 
> wrote:
>
> >Hi Pawel,
> >
> >The reason is the sort product can not use the EXCP access method with
the
> >compressed data set and instead chooses BSAM as the access method.
> >The EXCP access method usually reads or writes on a cylinder (or more)
> >boundary while BSAM, as its name suggests, reads or writes block by
block.
> >
> >Hope that helps,
> >Yifat Oren.
> >
> >-Original Message-
> >From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
> Behalf
> >Of Pawel Leszczynski
> >Sent: Wednesday, January 27, 2010 12:56 PM
> >To: IBM-MAIN@bama.ua.edu
> >Subject: why compression costs additional I/O?
> >
> >Hello everybody,
> >Recently we are reviewing our EndOfDay jobs looking for potential
> >performance improvements (reducing CPU/elapsed time).
> >We have several jobs sorting big datasets where output is
SMS-compressible
> >(type: EXTENDED) datasets.
> >When we compare such sorting with sorting on non-compressible output we
> can
> >see this:
> > EXCP   TCB   SRB   el.time
> >TESTXWP5   STEP110 00   757K   3.51.709.01 <-- w/o
> >compression
> >TESTXWP5   STEP120 00  1462K   3.62  2.89  10.45 <-- w.
compresion
> >
> >We guess that big SRB in (2) goes for compression (that we understand -
we
> >probably quit compression at all), but we don't understand 2 times
bigger
> >EXCP in second case.
> >
> >Any ideas will be appreciated,
> >Regards,
> >Pawel Leszczynski
> >PKO BP SA
> >
> >--
> >For IBM-MAIN subscribe / signoff / archive access instructions, send
email
> >to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the
> >archives at http://bama.ua.edu/archives/ibm-main.html
> >
> >--
> >For IBM-MAIN subscribe / signoff / archive access instructions,
> >send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> >Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.

Re: DFSORT memory used & paging

[David Betten]

Good points Kees.  The original question didn't indicate a paging
"problem",
just an increase in page DS use and demand paging.  So yes, tuning the
paging
subsystem is certainly a valid approach.  I was speaking mainly from a
DFSORT
perspective on ways to minimize paging.  Usually a small amount of demand
paging isn't a problem, but then again it depends on what workload is doing
it.




IBM Mainframe Discussion List  wrote on 01/15/2010
03:47:26 AM:


>
> Dave,
>
> I would have expected you to also mention a different approach: check
> the paging configuration.
>
> During the years I have become quite confident about the intelligence
> DFSORT has developped to use the available system resources, without
> causing problems in the system. One of the presumptions is the
> installation has a adequate paging configuration. We had a similar
> situation lately where we had paging performance problemns during DFSORT
> jobs. We have ample storage and the large DFSORT jobs decided to take a
> couple of GB central storage for its sort. This caused a large page out
> activity (like you described above) with I/O problems.
>
> After investigating the situation, the conclusion could only be that
> either DFSORT miscalculated the amount of available storage *or* the
> paging subsystem was not able to handle a substantial page-out load. We
> looked at our paging configuration and the spreading of the volumes over
> the internal ESS ranks and concluded that it contained several hotspots.
> We reconfigured the paging volumes and never saw the problem again.
>
> Everybody has read many years ago the recommandation for his/her paging
> configuration to spread it over as many volumes, channels and control
> units as possible, said "yes, of course", and continued with other
> things. A under-optimal paging configuration works well most of the time
> except in special situations like large dumps and yes, large DFSORTs.
>
> I think that also in Rez's case, both sides of the problem should be
> examined and if the paging configuration is insufficient *and* he is not
> able to correct this, the other option is to limit DFSORT's use of
> storage.
>
> Kees.
> **
> For information, services and offers, please visit our web site:
> http://www.klm.com. This e-mail and any attachment may contain
> confidential and privileged material intended for the addressee
> only. If you are not the addressee, you are notified that no part
> of the e-mail or any attachment may be disclosed, copied or
> distributed, and that any other action related to this e-mail or
> attachment is strictly prohibited, and may be unlawful. If you have
> received this e-mail by error, please notify the sender immediately
> by return e-mail, and delete this message.
>
> Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries
> and/or its employees shall not be liable for the incorrect or
> incomplete transmission of this e-mail or any attachments, nor
> responsible for any delay in receipt.
> Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> Dutch Airlines) is registered in Amstelveen, The Netherlands, with
> registered number 33014286
> **
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT memory used & paging

[David Betten]

>
> Recommend you start at the IBM DFSORT home-page where there are several
> documentation and supplemental technical references on DFSORT and
> memory usage.
>
> Scott Barry
> SBBWorks, Inc.
>
> Suggested Google advanced search argument on this topic:
>
> dfsort memory usage site:ibm.com
>

Good point Scott.  I should've included that in my original post.
There's also the DFSORT Tuning Guide which is very helpful.

Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT memory used & paging

[David Betten]

>
> DFSORT memory used & paging
>
> R Hey
>
> to:
>
> IBM-MAIN
>
> 01/14/2010 09:20 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> Hi Frank/List,
>
> My client has some very big SORT jobs.
> When a few run at the the same time, the page DS USE% increases
> (demanding paging).

This is likely due to DFSORT creating large memory objects, Hiperspaces
or Dataspaces for sorting in memory and that's causing pages to be
stolen from other workloads which then get paged out.

>
> Q1-
> How do I make sure BIG sort jobs don't use too many frames,
> when they are hard to get?

DFSORT has installation defaults that can limit the storage used
by all concurrent sorts.  EXPOLD controls the amount of "old" (not
recently referenced) frames that are considered in DFSORT's calculations
of storage available for sorting in memory.  The shipped
default is MAX.  You can change this if you want to limit that number of
old frames DFSORT uses which in turn would reduce the page stealing and
demand paging.
For example, if you set EXPOLD=0, then DFSORT would only consider available
frames
in its calculations of storage availabe for memory objects, Hiperspaces and
Dataspaces.

>
> Q2-
> Given the sort options we use, is it true that
> SORT will use TMAXLIM size of storage,
> then if data is too big, it will increase the storage use to max of DSA
> size?
> So it never goes over DSA size?

This is correct.

>
> Q3-
> If I get
> ICE199I 0 MEMORY OBJECT STORAGE USED = 3463M BYTES
> it means 3.4GB of data was sorted in memory, right?
>

Correct.

> Q4-
> If YES to both Q2/Q3, then how is 3GB of data is sorted in 64M of memory
> (our DSA)
> If, not, how does it work?

DFSORT will use up to 64MB of virtual storage for buffers, control blocks,
etc.  It can also create Hiperspaces or memory objects to be used as
intermediate storage for sorting the data.  The intermediate storage is
limited by DFSORT installation defaults such as EXPMAX, EXPOLD, HIPRMAX,
MOSIZE, DSPSIZE, etc.  It's also limited by the amount of available storage
at the time the sort executes.


>
> Q5-
> We use SIZE=MAX, but the job has SIZE=32259547 (it can be different for
> another run).
> Is this because the size of data being sorted determines the actual SIZE
> used?

Yes, DFSORT's Dynamic Storage Adjustment (DSA) alogorithm calculates the
optimal amount of main storage for a sort.  The DSA installation default is
a limit.  So DFSORT will use minimum of the calculated value or the DSA
limit.

>
>
> TIA,
> Rez
>
>
> Our sort option are:
>
> MODULE ICEAM1
>   -  -  -  -  -  -  -  -  -  -  -
> ARESALL0
> ARESINVNOT APPLICABLE
>   -  -  -  -  -  -  -  -  -  -  -
> DSA64
>   -  -  -  -  -  -  -  -  -  -  -
> DYNALOC(SYSALLDA,3)
>* (SYSDA,4)
>   -  -  -  -  -  -  -  -  -  -  -
> EXPMAX MAX
> EXPOLD MAX
> EXPRES 0
>   -  -  -  -  -  -  -  -  -  -  -
> HIPRMAXOPTIMAL
>   -  -  -  -  -  -  -  -  -  -  -
> MAXLIM 1048576
> MINLIM 450560
> MOSIZE MAX
>   -  -  -  -  -  -  -  -  -  -  -
> OVERRGN65536
> RESALL 4096
> RESET  YES
> RESINV NOT APPLICABLE
>   -  -  -  -  -  -  -  -  -  -
> SIZE   MAX
>   -  -  -  -  -  -  -  -  -  -
> TMAXLIM6291456
>   -  -  -  -  -  -  -  -  -  -
> VIONO
>
>
> joblog had:
>
> ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION
> MODULE
> SELECTED
> ICE088I 5 PAK328 .SORT305 . , INPUT LRECL = 650, BLKSIZE = 27950, TYPE =
> FB
> ICE093I 0 MAIN STORAGE = (MAX,32259547,32259547)
> ICE156I 0 MAIN STORAGE ABOVE 16MB = (32122707,32122707)
> ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0
> ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
>
> ICE128I 0 OPTIONS:
> SIZE=32259547,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=AB
> END,MS
> GDDN=SYSOUT
> ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO
> ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=
> (SCRATCH ,004),ABCODE=MSG
> ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109
> ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
>
> ICE131I 0 OPTIONS:
> TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DS
> A=64
> ICE132I 0 OPTIONS:
> VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE
> ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
> ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX
> ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
> ICE235I 0 OPTIONS: NULLOUT=RC0
> ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
> ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
> ICE750I 0 DC 3576062750 TC 0 CS DSVVV KSZ 36 VSZ 36
> ICE752I 0 FSZ=5501635 RC IGN=0 E AVG=652 0 WSP=4658983 C DYN=0
> 0
> ICE751I 1 BA-K22788 BB-K24705 BC-K24705 E8-K90013
> ICE090I 0 OUTPUT LRECL = 650, BLKSIZE = 27950, TYPE = FB (SDB)
> ICE080I 0 IN MAIN STORAGE SORT
> ICE055I 0 INSERT 0, DELETE 0
> ICE054I 0 RECORDS - IN: 5501623, OUT: 5501623
> ICE134I 0 NUMBER OF BYTES SORTED: 3576054950
> ICE165I 0

Re: "Sort Capacity Exceeded" thought / question

[David Betten]

John,
  The answer is that it's a little of both... "technical reason" and
"we didn't think of it".  We are looking at ways to improve our dynamic
allocation of work data sets to provide some additional flexibility.  So it
is on our list of things we'd like to improve on.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/21/2009
01:57:44 PM:

> [image removed]
>
> Re: "Sort Capacity Exceeded" thought / question
>
> McKown, John
>
> to:
>
> IBM-MAIN
>
> 12/21/2009 02:02 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> > -Original Message-
> > From: IBM Mainframe Discussion List
> > [mailto:ibm-m...@bama.ua.edu] On Behalf Of Lizette Koehler
> > Sent: Monday, December 21, 2009 12:47 PM
> > To: IBM-MAIN@bama.ua.edu
> > Subject: Re: "Sort Capacity Exceeded" thought / question
> >
> > John,
> >
> > What is your current parameter for dynamic SortWKxx set at?
> > Maybe you just need to increase that number?  Or if you have
> > one-off types, provide them with the SORT Over ride that
> > could let them allow for more dynamically allocated sortwk.
> >
> > Lizette
> >
>
> We have DYNALLOC set to 31. But this is in an ICEAM2 environment
> (internal sort). Apparently the application does not pass an
> accurate count to SORT. And, from what I can see, it does multiple
> internal sorts. I did state a JCL error, but the error was not
> having an SORTWKnn statements in the JCL at all.
>
> But that is not the prime reason for my inquiry. I am just curious
> if this facility is not in sort for a technical reason, a business
> reason, or a "we didn't think of that" reason.
>
> I do know how to fix it - we added a lot of large SORTWKnn DD statements.
>
> --
> John McKown
> Systems Engineer IV
> IT
>
> Administrative Services Group
>
> HealthMarkets(r)
>
> 9151 Boulevard 26 * N. Richland Hills * TX 76010
> (817) 255-3225 phone * (817)-961-6183 cell
> john.mck...@healthmarkets.com * www.HealthMarkets.com
>
> Confidentiality Notice: This e-mail message may contain confidential
> or proprietary information. If you are not the intended recipient,
> please contact the sender by reply e-mail and destroy all copies of
> the original message. HealthMarkets(r) is the brand name for
> products underwritten and issued by the insurance subsidiaries of
> HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-
> West National Life Insurance Company of TennesseeSM and The MEGA
> Life and Health Insurance Company.SM
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: IRLM timeout after IPL

[David Betten]

Did you change the size of the IRLM lock structure?  This is going back a
ways but I recall having a problem when the IRLM lock structure was too
large, it caused a loop when calculating the number of entries.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 09/29/2009
12:45:22 PM:

> [image removed]
>
> IRLM timeout after IPL
>
> Miklos Szigetvari
>
> to:
>
> IBM-MAIN
>
> 09/29/2009 12:46 PM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> Hi
>
> If someone has got this:
> After an IPL we got :
> DXR133I MRLM001 TIMEOUT DURING GLOBAL INITIALIZATION WAITING FOR NOEVENT
> and unable to start DB2 MQM
>
> Have opened a PMR, but  maybe someone has seen this.
>
> --
> Miklos Szigetvari
>
> Development Team
> ISIS Information Systems Gmbh
> tel: (+43) 2236 27551 570
> Fax: (+43) 2236 21081
>
> E-mail: miklos.szigetv...@isis-papyrus.com
>
> Info: i...@isis-papyrus.com
> Hotline: +43-2236-27551-111
>
> Visit our Website: http://www.isis-papyrus.com
> ---
> This e-mail is only intended for the recipient and not legally
> binding. Unauthorised use, publication, reproduction or
> disclosure of the content of this e-mail is not permitted.
> This email has been checked for known viruses, but ISIS accepts
> no responsibility for malicious or inappropriate content.
> ---
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: help with smf

[David Betten]

SMF type 15 is written for data sets that are opened for OUTPUT, UPDAT,
INOUT or OUTIN processing.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 08/04/2009
09:05:46 AM:

> [image removed]
>
> help with smf
>
> larry macioce
>
> to:
>
> IBM-MAIN
>
> 08/04/2009 09:07 AM
>
> Sent by:
>
> IBM Mainframe Discussion List 
>
> Please respond to IBM Mainframe Discussion List.
>
> I have a ps file that is getting changed and I need to find the job doing
> it.I need to know which smf records I need to pull to get the info. I've
> looked through the
> smf record types but it isn't jumping out at me.
> Any help is greatly appreciated
> thanks
> Larry
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT on z/OS 1.10 problem

[David Betten]

John,
  I think you should open a PMR and we'll research the cause of the
error.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 06/15/2009
11:02:59 AM:

> Well, I finally found something. If I put in a MOSIZE=0 on the OPTION
> statement in DFSPARM, then the problem goes away.
>
> Sorry, but I'm just not having a good week (too many weekend problems).
>
> --
> John
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Using FTP to send loadlib

[David Betten]

I usually just terse with TRSMAIN, ftp the tersed file, then unterse with
TRSMAIN.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 04/27/2009
01:32:25 PM:

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On
> Behalf Of Kurt Eastwood
> Sent: Monday, April 27, 2009 10:32 AM
> To: IBM-MAIN@bama.ua.edu
> Subject: Using FTP to send loadlib
>
> Hello,
> =A0
> I have been digging through the OS/390 TCP/IP OE: User's Guide and have
> not=
>  been able to get this to work.
> =A0
> Is it possible to send a loadlib from 1 mainframe to another mainframe
> usin=
> g FTP, either batch or interactive?
> =A0
> If so, would anyone be willing to share some batch jcl with me?
> =A0
> Thanks,
> Kurt=0A=0A=0A
>
> 
>
> I think you will have to unload it using IEBCOPY to a flat file, then
> send that as BIN. After receiving it, use IEBCOPY to load the PDS.
>
> OR, you can use TSO XMIT to a file, FTP as BIN, then point TSO RECEIVE
> to the file...
>
> Regards,
> Steve Thompson
>
> -- Opinions expressed by this poster may not reflect those of poster's
> employer. --
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SMF reporting question

[David Betten]

I'm not that familiar with it but the RMF Spreadsheet Reporter might povide
you with some z/OS performance reporting.
I'm pretty sure it comes with RMF so it probably meets your requirement of
being something you already have.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/26/2009
11:07:43 AM:

> I realize that most people likely use SAS for SMF reporting. We did this
in
> the past. Management has declared that SAS on the z is simply too
expensive.
> They also declared that installing the Windows version on our main user's
PC
> is also too expensive. End of discussion on that point.
>
> So, what else could be used? It must either be something that we already
> have, such as COBOL, EasyTrieve Plus (without the SMF add on), REXX,
HLASM,
> ... . We cannot spend any "hard money" on this. Oh, it would be nice if
if
> were very CPU efficient because we just downgraded our z9BC from a V02 to
a
> T02 in order to save on software costs. And it must be such that doing
"ad
> hoc" requests can be responded to quickly. Yes, I know, "give me the
world,
> but don't spend any money".
>
> I am actually looking at downloading the raw SMF data to a Linux box (my
> desktop) using BINary and SITE RDW. I did this for last week's SMF and
had
> about 14Gib of data. I know how to read this with Java. This may actually
be
> what I end up looking at doing. But I am the only person in my group who
is
> even mildly Java literate. The main performance person is not. And he
> doesn't have access to my PC anyway. Of course, that is one reason that
I'm
> looking at Java. I have written Java in the past (minor application)
which
> truly was "run anywhere". At least it ran, as compiled on the Linux box,
on
> 32 bit Linux/Intel, 64 bit Linux/Intel, Mac OS/X, 32 bit Windows, and on
the
> z. I just transferred the jar file and ran it.
>
> Any thoughts or commiserations appreciated.
>
> --
> John
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT statement sequence

[David Betten]

> Thanks for your kind answer. Does that means that AFTER reading all the
> matching input records, that subset is THEN sorted?

Correct.  As each record is processed, the INCLUDE criteria is evaluated.
If a the record does not meet the INCLUDE criteria it is discarded and not
included in the sort.


> The reason is I have to select a wide variety of different inputs
toproduce a
> subset and THEN sort that lot for reporting. My goal: to have as fewbytes
to
> sort after selecting what I need with one pass of the input dataset.
>
> I wonder, if I do all my COPY and using INCLUDE statements and then do
this
> two stage method:
>
> COPYFROM(INPUT) TO(TEMP1) USING()
> COPYFROM(TEMP1) TO(TEMP2) USING()
>
> Will this work better for large input datasets?


This is one of those "it depends" types of situations.  If a large
percentage of the records are likely to be included, I think you are fine
with just doing 1 sort with the include statements.  If a small percentage
are likely to be included, you may be better off doing the copy with
INCLUDE followed by the sort.

The reason I say this is that when sorting with INCLUDE, DFSORT has no way
of knowing how many records will match the criteria.  Therefore, we need to
allocate resources (main storage, work space, etc.) under the assumption
that all records will be included.  So you may have some wasted resources
if only a small percentage of the records are actually included.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT statement sequence

[David Betten]

It is FALSE.  All of the control statements are read at initialization and
the include fields are evaluated during the input phase of the sort to
select only the records that meet the criteria.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/09/2009
07:52:51 AM:

> Good day
>
> Where is it documented that the sequence of sort control statement does
> play a role in performance? My searches on Google, IBM-MAIN and DFSORT
> couldn't produce something 'hard and fast'...
>
> My gut feeling is that
>
> INCLUDE COND=(1,8,CH,EQ,C'ABCDEFGH')
> SORT FIELDS=(1,8,CH,A)
>
> is better over this
>
> SORT FIELDS=(1,8,CH,A)
> INCLUDE COND=(1,8,CH,EQ,C'ABCDEFGH')
>
> for very LARGE input.
>
> Is this true or false? (If false, I can always do a COPY with
> INCLUDE and then
> do my SORT FIELDS)
>
> Thank you in advance.
>
> Groete / Greetings
> Elardus Engelbrecht
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: hiperspace, DSNUTILB and df/sort

[David Betten]

I'm not entirely sure I understand your question but I'll try to explain
how DFSORT uses Hiperspace and maybe that will help.

As part of DFSORT initialization, the characteristics of the sort are
evaluated along with available resources and a decision is made about what
type and how much storage to use.

If a decision was made to use Hiperspace storage than a Hiperspace is
created.  At this point the Hiperspace is empty which may be why you see it
as "swapped out".

As the DB2 utility passes input records to DFSORT, they are written to the
Hiperspace.  I'm guessing this filling of the Hiperspace might be what is
looking like page ins to you.

Once all input records have been processed, DFSORT then begins to read the
records back from the Hiperspace and pass them back to the DB2 Utility in
sequence.  Normally we use RELEASE=YES so that the pages are released as
they are read.

At completion, DFSORT then deletes the Hiperspace.




Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
1-301-556-0943
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/23/2009
01:32:12 PM:

> We are trying to understand how hiperspace is being used. z/os 1.9.
Using
> DSNUTILB and it's invocation of df/sort, it appears that when the
hiperspace
> is initially created, it is immediately swapped out, then demand paged
back
> in. When I look at type30 records, or in MON3 we see pagins, but never
any
> page outs, nor any swap outs during the step duration. in MON3, we see
> sometimes 1K's worth of pages in DIV status, but again, we never see any
> page/swap outs. What am I missing or where should I be looking ?
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Is there a tool to put SMF data into DB/2?

[David Betten]

I believe IBM's Tivoli Decision Support for z/OS collects SMF data into DB2
tables for analysis and reporting.  I think this tool used to be called
EPDM.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/17/2008
07:30:57 AM:

> I was poking around in the manuals and saw some things about possibly
> putting the SMF data related to RACF into DB/2 tables, and it got me
> wondering. Is there a tool that will convert *all* SMF record types and
put
> them into DB/2 tables? That would be awesome if there were.
>
>
>
>
>
>
>
> David Logan
>
> Manager of Product Development, Pitney Bowes Business Insight
>
>   http://centrus.com
>
> W: (720) 564-3056
>
> C: (303) 818-8222
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Finding the correct IEASYSxx member

[David Betten]

I think if you enter D IPLINFO from the console it gives you the IEASYS
list.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 10/15/2008
02:55:21 PM:

> How can I trace back and find out which IEASYSxx member the system
> was IPL'ed with?
>
> If you don't key in a member name at IPL time doesn't the system
> pick IEASYS00?
>
> Thanks.
> _
> LEGAL NOTICE
> Unless expressly stated otherwise, this message is confidential
> and may be privileged. It is intended for the addressee(s) only.
> Access to this E-mail by anyone else is unauthorized.
> If you are not an addressee, any disclosure or copying of the
> contents of this E-mail or any action taken (or not taken) in
> reliance on it is unauthorized and may be unlawful. If you are not an
> addressee, please inform the sender immediately, then delete this
> message and empty from your trash.
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Opinions? Subset input with DFSORT before processing in REXX?

[David Betten]

I would use DFSORT to extract the 0200 records and have your REXX read the
smaller file.  Your REXX is probably using EXECIO and DFSORT will be able
to read through that entire file faster and use much less cpu time.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 05/23/2008
10:25:07 AM:

> I would, definitely.
>
> You can do some Wonderful Things with DFSORT/ICETOOL these days. The
> SPLICE operator is particularly wonderful for RACF database unloads
> as you can join up base userid records with their segments, so you
> can then handle single records at a time in a report instead of
> having to go doing subsequent reads for the segments.
>
> I love Frank Yaeger. He's a God!
>
> ;-)
>
> Brian
>
>
> -
> Email sent from www.virginmedia.com/email
> Virus-checked using McAfee(R) Software and scanned for spam
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Long running Batch programs keep IMS databases offline

[David Betten]

My knowledge of IMS is minimal but I seem to recall there being two flavors
of IMS batch processing

IMS DL/I was usually offline processing where files could not be accessed
concurrently by the onlines

IMS BMP (Batch Message Processing) was where the batch jobs accessed the
data bases through message processing regions similar to online transaction
processors so they could run concurrent with online.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 05/06/2008
04:09:35 PM:

> It's been a decade or two, but concurrent batch/online was a base
> feature of IMS. Problems included the difficulty in backing out the
> effects of a malfunctioning batch program.
>
>
>
>
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Anne & Lynn Wheeler
> Sent: Monday, May 05, 2008 11:29 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: Long running Batch programs keep IMS databases offline
>
> The following message is a courtesy copy of an article
> that has been posted to bit.listserv.ibm-main as well.
>
>
> "F" <[EMAIL PROTECTED]> writes:
> > We have IMS 9 on z/OS and I am fairly new to the platform and have a
> > vested interest in fixing it. Every night we have batch programs that
> > run which in return keeps our databases offline for a long time and as
> > a result our applications are not available for processing.
> >
> > I want to know why batch programs and databases cannot both be online
> > at the same time ? If the batch programs read the databases, then why
> > are they offline ?
> >
> > Anyways, what are some ways of ensuring that batch jobs and databases
> > can both run and be online at the same time ?
>
> some recent discussions about "overnight batch window" ... which
> requires exclusive access to all the information ... as opposed to
> "online".
> http://www.garlic.com/~lynn/2008b.html#74 Too much change opens up
> financial fault lines
> http://www.garlic.com/~lynn/2008c.html#92 CPU time differences for the
> same job
> http://www.garlic.com/~lynn/2008d.html#30 Toyota Sales for 2007 May
> Surpass GM
> http://www.garlic.com/~lynn/2008d.html#31 Toyota Sales for 2007 May
> Surpass GM
> http://www.garlic.com/~lynn/2008d.html#73 Price of CPU seconds
> http://www.garlic.com/~lynn/2008d.html#87 Berkeley researcher describes
> parallel path
> http://www.garlic.com/~lynn/2008d.html#89 Berkeley researcher describes
> parallel path
> http://www.garlic.com/~lynn/2008g.html#55 performance of hardware
> dynamic scheduling
> http://www.garlic.com/~lynn/2008h.html#50 Microsoft versus Digital
> Equipment Corporation
>
> there were some number of efforts in the 90s (billions of dollars) that
> looked at business process re-engineering to leverage killer micros and
> distributed object-oriented technology to implement "straight-through
> processing" (eliminating the "overnight batch window"). It turns out
> that many of these had grandious failures when nobody bothered to do any
> speeds&feeds until very late in the effort ... frequently belatedly
> discovering that the distributed object-oriented technology had a factor
> of 100 times increase in overhead (compared to the typical Cobol batch
> implementation), totally obliterating any hopes of throughput
> improvements.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> NOTICE: This electronic mail message and any files transmitted with
> it are intended
> exclusively for the individual or entity to which it is addressed.
> The message,
> together with any attachment, may contain confidential and/or
> privileged information.
> Any unauthorized review, use, printing, saving, copying, disclosure
> or distribution
> is strictly prohibited. If you have received this message in error,
please
> immediately advise the sender by reply email and delete all copies.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Identify matching records

[David Betten]

Just remember that using ALLDUPS to do this assumes there aren't already
dupes in either of the files.  I usually run the inputs through ICETOOL to
drop dupes first and then use those files as input to ALLDUPS.

//STEP1EXEC PGM=ICETOOL
//TOOLMSG   DD SYSOUT=*
//DFSMSGDD SYSOUT=*
//INDD DISP=SHR,DSN=INPUT1
//OUT   DD DISP=(,CATLG),DSN=INPUT1.NODUPES,
// SPACE=(CYL,(100,100),RLSE,UNIT=SYSDA,DCB=*.IN
//TOOLINDD *
 SELECT FROM(IN) TO(OUT) ON(01,10,CH) FIRST
/*
//*

The use INPUT1.NODUPES and INPUT2 as input to the example below.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 04/25/2008
03:05:54 PM:

> Thanks, I did go looking for examples in the DFHSORT web site but
> couldn't locate them. What is this link?
>
> Thanks
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Tony B.
> Sent: Friday, April 25, 2008 12:04 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: Identify matching records
>
> From various examples in the DF/SORT web site.
>
>
>
>
>
> //STEP1EXEC PGM=ICETOOL
> //TOOLMSG   DD SYSOUT=*
> //DFSMSGDD SYSOUT=*
> //I DD DISP=SHR,DSN=INPUT1
> //  DD DISP=SHR,DSN=INPUT2
> //MATCH DD SYSOUT=*
> //NONMATCH  DD SYSOUT=*
> //TOOLINDD *
>  SELECT FROM(I) TO(MATCH) ON(01,10,CH) ALLDUPS DISCARD(NONMATCH)
>
>
> matching on columns 1-10.
>
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
Behalf
> Of George, William (DHCS-ITSD)
> Sent: Friday, April 25, 2008 1:06 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Identify matching records
>
> We have an IMMEDIATE need to identify records in two files that match.
>
> One file has several million records while the other has about ½ million.
> We need to locate all records in the smaller file that are also in the
> larger file ASAP.
>
>
>
> I pose this to the group just in case someone can identify a means via
SORT,
> SUPERC or some other means faster than I can research to find an existing
> method or building one.
>
>
>
> Thank You for any insights.
>
>
>
> Bill
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date:
4/25/2008
> 7:42 AM
>
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date:
4/25/2008
> 7:42 AM
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.524 / Virus Database: 269.23.4/1397 - Release Date:
4/25/2008
> 7:42 AM
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SMS allocation in Cylinders using AMS

[David Betten]

I'd also ask if when you see the dataset using 3 times what the user
requested, are you looking immediately after it's been defined or after
data has been loaded into it?  It might be extending once it's populated.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 04/16/2008
01:52:52 PM:

> Gilbert,
>
> What is your device geomtery on your CDS BASE DISPLAY? For 3390 it
> should be 56664 bytes per track and 15 tracks per cyl. If it were
> set to 3380, I would expect your allocations to be approx 20%
> smaller than requested.
>
> Where in the manual did it tell you not to use Cyls? I've been using
> Cyls or Tracks for years with no problem.
>
> 43 cyls for 30 MB is about right. Might depend on your Cisize.
>
> 
>
> From: Gilbert CardenasSent: Wed 4/16/2008 1:28 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: SMS allocation in Cylinders using AMS
>
>
>
> Hello everyone, I have an unknown that I have not been able to figure out
or
> at least I don't know where to look.
> I have a user who is trying to define a vsam file using IDCAMS as
follows:
>
> DEFINE CLUSTER -
>( NAME(BLAH.BLAH.BLAH) -
> LINEAR -
> REUSE -
> CYL(30) -
> SHAREOPTIONS(3 3) ) -
>DATA -
>( NAME(BLAH.BLAH.BLAH.DATA) -
>   ) -
>   CATALOG(BLAH)
>
> The problem is, when I look at the dataset (which is SMS managed) it
usually
> ends up being around 3 times larger than what the user requested.
>
> Of course the first place I looked was in the dataclas that this dataset
gets
> assigned and the only thing that the dataclas specifies is a volume
> count of 1.
> There are no overrides anywhere else for primary or secondary space.
>
> Looking at the Access Method Services documentation, it states that you
> should not use the TRACKS or CYLINDERS parameters. If you use them for an
> SMS-managed data set, space is allocated on the volumes selected by SMS
in
> units equivalent to the device default
> geometry.
>
> Could the reason that this dataset is being allocated larger than
requested
> have to do with the fact that the sms volumes are mod 9 volumes?
>
> Btw, I tried allocating the dataset in Megabytes (30) and I got 43 cyl
with 1
> extent.  I don't know what the conversion of megabytes to cylinders is
but
> this still doesn't seem right.
>
> Regards,
> Gil.
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: how fast can I sort on mainframe (using DFSORT)?

[David Betten]

Chris,
  Since I joined the team as the performance lead a couple years ago,
Frank now defers these types of questions to me.  Everything you (and all
the others that responded) have said are good points about DFSORT
performance.  But the multitude of recommendations just reiterates the
point in my original response that there are so many factors, you could
spend weeks tuning things that don't improve anything.  Just as I preached
during my 15+ years of batch tuning, I still say that before you start
tuning anything, you have to determine where you are spending time.  Then
you start your tuning there.  I always prefer to get some data (sysouts,
SMF, etc.) before I start making recommendations.  That is why I asked
Pawel to send us a few sample sysouts as a start.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/10/2008
12:48:05 AM:

> Pawel,
>
> I have been out of the sorting business for a while, but the target you
> can go for is what I call 2T, where T is the time to read in the data.
>
> The 2T comes from the fact that you have to read it in and then write it
> out.
>
> Writing to sort work files should not be the problem, hopefully the sort
> is writing to several of them in parallel during the sort phase and
> reading from them in parallel during the merge phase.
>
> EXCP's are an accountant's measurement, not a tuner's measurement.  Look
> at your SMF data and see what the channel connect times are for SORTIN
> and SORTOUT.  If the elapsed time is significantly greater than those
> two added together then you have to start looking for the bottleneck.
>
> My first reaction to your 17M/second was a question.  Is that data
> sorted divided by total time?  I assume it is.  If so then you are
> getting about 34M/second on input and output, and that is not a bad
> number.
>
> Tape and DASD inputs and output will give you different numbers.  There
> tends to be more interference with tape files.  Fewer channels, no PAV.
>
> Back when DASD was real 3390, 4M/second was the target, so you are doing
> pretty good.  I couch that with the fact that I have not run throughput
> tests on the various DASD subsystems available today.  34M may be great
> or it may be just average.  I don't know.
>
> The best thing you can do is give each sort enough memory.  You will
> need to ask Frank Yeager what the target amounts of memory are for a 10G
> sort.  My guess is about 17M to 24M.  A 100G sort might need 50M to 75M.
> A 1G sort might need 5M to 7M.
>
> What you are attempting to do is make it so there is no need to do an
> intermediate merge, or in the case of blockset sorting, that the final
> merge remains efficient.
>
> I would not attempt to get a 10G sort to run as a turnaround or incore
> sort.  It will put a large strain on RSM/ASM and not buy you that much.
>
> I hope Frank weighs in on this.
>
> Chris Blaicher
> BMC Software, Inc.
> These are my own personal comments and do not reflect the views of my
> employer.  Please do your own evaluation of any statements or
> suggestions made.
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Pawel Leszczynski
> Sent: Sunday, March 09, 2008 4:06 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: how fast can I sort on mainframe (using DFSORT)?
>
> Hello everybody,
>
> I realize subject is VRY broad and my question VRY general,
> but...
>
> Recently in our shop we are reviewing our whole batch processing.
> Most of the time of EOD processing is consumed by sort of many big
> sequential
> files. (One such file has approximately order of 10GB, 10mln records)
> I listed few tens of such batch jobs (the longest-lasting ones)
> and computed mean sorting rate.
> It appeared to be about 1GB/min ~ 17MB/sec
>
> I suppose it's very poor result(???).
>
> Can you tell me how much I can improve this?
>
> These batch jobs are little CPU-consuming (~10% of one CPU),
> I suppose that major concern is to:
>
> -limit EXCPS (1)
> -increase throughput rate from DASD to central storage (2)
>
> I realize that sorting whole file in central storage (hiperspace
> sorting) would
> eliminate need to use work files and EXCPS to them.
> How much central storage is needed to handle in-storage sorting for
> let's say 10 GB file???
>
> How much can I  improve (2)?
>
> Can you tell me what is mean sorting rate in your installation?
> Before starting I would just like to know if I can achieve substantial
> improvement.
>
> TIA,
> Pawel Leszczynski
> PKO BP SA
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
---

Re: how fast can I sort on mainframe (using DFSORT)?

[David Betten]

Pawel,
  It's really hard to say what is a "good" sorting rate.  There are so
many factors that can impact the performance of the sort.  File
characteristics, i/o contention, storage resources, etc.  I have seen
customer sorts with much higher throughput rates than you and I've seen
some with worse.  The DFSORT team is alwasy willing to assist with tuning.
If you can send a note to our hotline ([EMAIL PROTECTED]) we can assist
you.  If possible, add //SORTDIAG DD DUMMY to some of the long running sort
steps and send us the sysout.  Be sure to send the entire sysout and not
just the DFSORT messages.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/09/2008
05:05:33 PM:

> Hello everybody,
>
> I realize subject is VRY broad and my question VRY general,
but...
>
> Recently in our shop we are reviewing our whole batch processing.
> Most of the time of EOD processing is consumed by sort of many big
sequential
> files. (One such file has approximately order of 10GB, 10mln records)
> I listed few tens of such batch jobs (the longest-lasting ones)
> and computed mean sorting rate.
> It appeared to be about 1GB/min ~ 17MB/sec
>
> I suppose it's very poor result(???).
>
> Can you tell me how much I can improve this?
>
> These batch jobs are little CPU-consuming (~10% of one CPU),
> I suppose that major concern is to:
>
> -limit EXCPS (1)
> -increase throughput rate from DASD to central storage (2)
>
> I realize that sorting whole file in central storage (hiperspace
> sorting) would
> eliminate need to use work files and EXCPS to them.
> How much central storage is needed to handle in-storage sorting for
> let's say 10 GB file???
>
> How much can I  improve (2)?
>
> Can you tell me what is mean sorting rate in your installation?
> Before starting I would just like to know if I can achieve substantial
> improvement.
>
> TIA,
> Pawel Leszczynski
> PKO BP SA
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT anomaly?

[David Betten]

I would change the dataset names instead of changing the copy to a sort.
Today, probably not much difference but if the file size grows over time,
the copy is going to be faster and require less resources than a sort.  I'd
change the following

Add a DD5 to your initial DEL005 step with a new data set named something
like PECT.TBC.SE.ALC.TEMP.IMPOUT
Change NCP0511 step to create PECT.TBC.SE.ALC.TEMP.IMPOUT
Change IDCAM020 step's FILEIN to PECT.TBC.SE.ALC.TEMP.IMPOUT
Change SORT01 step's SORTIN to PECT.TBC.SE.ALC.TEMP.IMPOUT

You could also add another BR14 step right after the SORT01 step to delete
the TEMP.IMPOUT if you want to avoid that extra copy occupying space.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/22/2008
11:04:26 AM:

> Thanks to everyone for replying to this. I am definitely going to get
> the apps guys to change the JCL. I just haven't whether it would be to
> change it to a sort, or create a second dataset. Will read the
> recommendations between the lines of the DFSORT manual...
>
> Regards
>
> Herbie
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Frank Yaeger
> Sent: 22 Januarie 2008 03:50 nm
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: DFSORT anomaly?
>
> David Betten wrote on 01/22/2008 06:53:52 AM:
> > I haven't seen any job output yet but I wanted to share this
> information
> > from DFSORT Application Programming Guide
> >
> > ***
> > For a copy application, the SORTIN data set should not be the same as
> the
> > SORTOUT data set or any OUTFIL data set because this can cause lost or
> > incorrect data or unpredictable results.
> > ***
> >
> > Frank probably knows more about this but it makes sense to me why we
> could
> > have problems for COPY.  When we do a SORT, all of the records from
> SORTIN
> > need to be read before we can write anything to SORTOUT.  This is
> because
> > that last record read could potentially be the first record written.
> > However, when we do a COPY, we can start writing to SORTOUT
> immediately.
> > I would just try changing your sort step to write a new dataset and
> see
> > what the results are.
>
> Dave has it right.  I just got here and was going to write a similar
> post.
> You SHOULD NOT use the same data set for SORTIN and SORTOUT for a COPY
> for
> the reason Dave states that we are doing parallel read/write.  Depending
> on
> how the data is arranged, you may get away with it, but you're taking a
> risk.
> We can't guarantee that you won't lose records.  So just don't do it.
>
> You can use the same data set for SORTIN and SORTOUT for a SORT because
> we
> don't do parallel read/write.  This is commonly known as a "suicide
> sort"
> because the "input" data set is overwritten so if something goes wrong
> (e.g. an I/O error) you don't have the input data set anymore.  So you
> would want to be sure you have a backup for the input data set if you do
> a suicide sort.
>
> Frank Yaeger - DFSORT Development Team (IBM) - [EMAIL PROTECTED]
> Specialties: PARSE, JFY, SQZ, ICETOOL, IFTHEN, OVERLAY, Symbols,
> Migration
>
>  => DFSORT/MVS is on the Web at http://www.ibm.com/storage/dfsort/
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
> Elavon Financial Services Limited
> Registered in Ireland: Number 418442
> Registered Office: Block E, 1st Floor, Cherrywood Business Park,
> Loughlinstown, Co. Dublin, Ireland
> Directors: Robert Abele (USA), John Collins,  Terrance Dolan (USA),
> Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson
> Elavon Financial Services Limited, trading as Elavon, is regulated
> by the Financial Regulator
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT anomaly?

[David Betten]

I haven't seen any job output yet but I wanted to share this information
from DFSORT Application Programming Guide

***
For a copy application, the SORTIN data set should not be the same as the
SORTOUT data set or any OUTFIL data set because this can cause lost or
incorrect data or unpredictable results.
***

Frank probably knows more about this but it makes sense to me why we could
have problems for COPY.  When we do a SORT, all of the records from SORTIN
need to be read before we can write anything to SORTOUT.  This is because
that last record read could potentially be the first record written.
However, when we do a COPY, we can start writing to SORTOUT immediately.
I would just try changing your sort step to write a new dataset and see
what the results are.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/22/2008
08:36:49 AM:

> What confused me though was why could IDCAMS copy the whole file to a
> backup?
>
> Regards
> Herbie
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Van Dalsen, Herbie
> Sent: 22 Januarie 2008 01:20 nm
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: DFSORT anomaly?
>
> That was my initial thinking, but I did not want to cloud anyone else's
> thinking...
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Shedlock, George
> Sent: 22 Januarie 2008 01:05 nm
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: DFSORT anomaly?
>
> Is it possible that the COBOL program is not closing the file and
> therefore any additional extents are not reflected in the VTOC / Catalog
> entry?
>
>
> George Shedlock Jr
> AEGON Information Technology
> AEGON USA
> 502-560-3541
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Van Dalsen, Herbie
> Sent: Tuesday, January 22, 2008 5:59 AM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: DFSORT anomaly?
>
> Hi all,
>
> WE are running z/OS 1.6 with no plans to upgrade because we aren't going
> to fork out any more for Tivoli 8.3. We also have Enterprise COBOL.
>
> This is the problem...
>
> The file was allocated Cyl,(15,5) and 16 records written to it by
> the COBOL program, it thus expanded to 3 volumes and lots of extents...
> In the same job, this file was sorted, but only 45900 records were
> sorted, into itself, and the rest of the records were lost.
>
> We have tried to re-create the problem by creating the multivolume file
> with IEBGENER... unsuccessfully...
>
> My thinking is this, and I have not found a MVS APAR stating that this
> might be the case, but here goes... Because of some problem / limitation
> in the COBOL / MVS File close routines... All three extents of the file
> are not known to DFSORT 3 steps lower and only at the actual end of the
> job, but how do I prove this.
>
> This occurred 3 times now, until we changed the allocation to
> Cyl,(200,50) and end up with the file on 1 volume, problem sorted?
>
> Any ideas, Any help would be greatly appreciated.
>
> Regards
>
> Herbie
> Elavon Financial Services Limited
> Registered in Ireland: Number 418442
> Registered Office: Block E, 1st Floor, Cherrywood Business Park,
> Loughlinstown, Co. Dublin, Ireland
> Directors: Robert Abele (USA), John Collins,  Terrance Dolan (USA),
> Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson Elavon
> Financial Services Limited, trading as Elavon, is regulated by the
> Financial Regulator
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send
> email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search
> the archives at http://bama.ua.edu/archives/ibm-main.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
> Elavon Financial Services Limited
> Registered in Ireland: Number 418442
> Registered Office: Block E, 1st Floor, Cherrywood Business Park,
> Loughlinstown, Co. Dublin, Ireland
> Directors: Robert Abele (USA), John Collins,  Terrance Dolan (USA),
> Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson
> Elavon Financial Services Limited, trading as Elavon, is regulated by
> the Financial Regulator
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
> Elavon Financial Services Limited
> Registered in Ireland: Number 418442
> Reg

Re: DFSORT anomaly?

[David Betten]

Herbie,
  Send the sysout from your job directly to me ([EMAIL PROTECTED]) and
I'll take a look.  We may need you to open an ETR on this but let me take a
look first.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/22/2008
07:23:16 AM:

> Kees,
>
> What bugs me is the following...
> Whether we have 90,000 of 160,000 records, the final number remains
>
> Display from COBOL PGM... T735 WRITTEN = 050634
> Display from DFSORT...RECORDS - IN: 45900, OUT: 45900
>
> Display from COBOL PGM... T735 WRITTEN = 052687
> Display from DFSORT...ICE054I 0 RECORDS - IN: 45900, OUT: 45900
>
> Regards
>
> Herbie
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Vernooy, C.P. - SPLXM
> Sent: 22 Januarie 2008 12:16 nm
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: DFSORT anomaly?
>
> "Vernooy, C.P. - SPLXM" <[EMAIL PROTECTED]> wrote in message
> news:<[EMAIL PROTECTED]
> t>...
> > "Van Dalsen, Herbie" <[EMAIL PROTECTED]> wrote in message
> >
> news:<[EMAIL PROTECTED]
> > >...
> > > Kees,
> > >
> > > That is actually that happens...
> > > Step1 creates the file,
> > > Step2 Makes a backup, using IDCAMS, and the backup does have 16000
> > > records, and step3 does the sort...
> > > very weird...
> > >
> > > //DEL005   EXEC PGM=IEFBR14
> > > //DD3  DD   DSN=HUBRECHT.SORTINOUT,
> > > // DISP=(MOD,DELETE,DELETE),
> > > // UNIT=SYSDA,SPACE=(TRK,0)
> > > //*
> > > //NCPO511  EXEC PGM=NCPO511
> > > //SYSOUT   DD   SYSOUT=*
> > > //SYSPRINT DD   SYSOUT=*
> > > //IPMOUT   DD   DSN=HUBRECHT.SORTINOUT,
> > > // DISP=(,CATLG,DELETE),
> > > // UNIT=SYSDA,SPACE=(CYL,(15,5)),
> > > // DCB=(RECFM=VB,LRECL=4504,BLKSIZE=0)
> > > //*
> > > //IDCAM020 EXEC PGM=IDCAMS
> > > //SYSOUT   DD   SYSOUT=*
> > > //SYSPRINT DD   SYSOUT=*
> > > //FILEIN   DD   DSN=HUBRECHT.SORTINOUT,DISP=SHR
> > > //FILEOUT  DD   DSN=HUBRECHT.SORTINOUT.BACKUP(+1),
> > > // DISP=(,CATLG,DELETE),
> > > // UNIT=SYSDA,
> > > // SPACE=(CYL,(15,5),RLSE),
> > > // DCB=(RECFM=VB,LRECL=4504,BLKSIZE=0)
> > > //SYSINDD   *
> > > REPRO INFILE(FILEIN) OUTFILE(FILEOUT) -
> > >   REPLACE
> > > //*
> > > //SORT01   EXEC PGM=SORT
> > > //SYSOUT   DD  SYSOUT=*
> > > //SYSPRINT DD  SYSOUT=*
> > > //SORTIN   DD  DSN=HUBRECHT.SORTINOUT,DISP=SHR
> > > //SORTOUT  DD  DSN=HUBRECHT.SORTINOUT,DISP=SHR
> > > //SYSINDD  *
> > > OPTION COPY
> > > INREC IFTHEN=(WHEN=(5,2,ZD,EQ,1,AND,1426,2,CH,EQ,C'48'),
> > > OVERLAY=(132:C'5',134:C'S',888:C'CT6')),
> > > IFTHEN=(WHEN=(5,2,ZD,EQ,1,AND,150,4,ZD,EQ,6010),
> > > OVERLAY=(54:+12,TO=ZD,LENGTH=2))
> > >
> > > Regards
> > >
> > > Herbie
> >
> > I really wonder if it possible to make Sort to this, with sortin and
> > sortout being the same dataset. Make sortout a different dataset and
> see
> > what happens. And/or check the manual for the validity of this
> sortstep.
> >
> > Kees.
>
> Or wait a few hours until the DFSort support team has woken up and read
> their mail.
>
> Kees.
> **
> For information, services and offers, please visit our web site:
> http://www.klm.com. This e-mail and any attachment may contain
> confidential and privileged material intended for the addressee
> only. If you are not the addressee, you are notified that no part
> of the e-mail or any attachment may be disclosed, copied or
> distributed, and that any other action related to this e-mail or
> attachment is strictly prohibited, and may be unlawful. If you have
> received this e-mail by error, please notify the sender immediately
> by return e-mail, and delete this message.
>
> Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries
> and/or its employees shall not be liable for the incorrect or
> incomplete transmission of this e-mail or any attachments, nor
> responsible for any delay in receipt.
> Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> Dutch Airlines) is registered in Amstelveen, The Netherlands, with
> registered number 33014286
> **
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
> Elavon Financial Services Limited
> Registered in Ireland: Number 418442
> Registered Office: Block E, 1st Floor, Cherrywood Business Park,
> Loughlinstown, Co. Dublin, Ireland
> Directors: Robert Abele (USA), John Collins,  Terrance Dolan (USA),
> Pamela Joseph (USA), Declan Lynch, John McNally, Malcolm Towlson
> Elavon Financial Services Limited, tra

Re: DFSORT question

[David Betten]

There can be a slight decrease in performance when you increase the number
of sortworks.  Going from say 32 to 48 or 64 probably won't have a major
impact.  However, jumping to something like 128 or 255, you will see a more
noticable impact.  If you want to send me the sysout from the sort, I can
take a look and see if there's any additional tuning we could do to improve
performance and/or decrease the sortwork requirement.




Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/14/2008
09:30:08 AM:

> This just came up. Which is better: to allocate more, smaller, SORTWKnn
> DD statements, or fewer, larger, SORTWKnn DD statements, or does it not
> really matter. By "better", I mean: "Which will result in a shorter run
> time?".
>
> The question came up due to a huge sort this weekend. DFSORT wanted to
> allocate the SORTWKnn space > 3390-3 volume. We only have 3390-3
> volumes. So the question is, do I update defaults to have more than 32
> volumes or do we consider creating 3390-9 volumes for SORTWKnn?
>
> This is IBM's DFSORT on z/OS 1.8. Yes, I am aware of Syncsort's MAXSORT
> capability. No, it will not be considered. End of discussion on that
> point, please.
>
> --
> John McKown
> Senior Systems Programmer
> HealthMarkets
> Keeping the Promise of Affordable Coverage
> Administrative Services Group
> Information Technology
>
> The information contained in this e-mail message may be privileged
> and/or confidential.  It is for intended addressee(s) only.  If you are
> not the intended recipient, you are hereby notified that any disclosure,
> reproduction, distribution or other use of this communication is
> strictly prohibited and could, in certain circumstances, be a criminal
> offense.  If you have received this e-mail in error, please notify the
> sender by reply and delete this message without copying or disclosing
> it.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Extracting SMF Records to find the userid who deleted a dataset

[David Betten]

Frank Yaeger just pointed out to me that the trailing blanks are not
necessary since DFSORT will pad the constant on the right with blanks to
the length of the field.  So you can just code it like this

//SORT  EXEC PGM=SORT,REGION=5000K
//SYSPRINT DD  SYSOUT=*
//SYSOUT   DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=inputsmf
//SORTOUT  DD  DISP=(NEW,PASS),DSN=&&TEMP,
// SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG
//SYSINDD   *
   SORT FIELDS=COPY
   INCLUDE COND=(6,1,BI,EQ,X'11',&,
   45,44,CH,EQ,
   C'deleted.dsn')
   OPTION VLSHRT
/*
//PRINTEXEC PGM=IDCAMS
//IN1  DD  DISP=(OLD,PASS),DSN=&&TEMP
//SYSPRINT DD SYSOUT=*
//SYSINDD   *
 PRINT INFILE(IN1) DUMP
/*





Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/13/2007
12:45:18 PM:

> You can try this
>
> //SORT  EXEC PGM=SORT,REGION=5000K
> //SYSPRINT DD  SYSOUT=*
> //SYSOUT   DD  SYSOUT=*
> //SORTIN   DD  DISP=SHR,DSN=inputsmf
> //SORTOUT  DD  DISP=(NEW,PASS),DSN=&&TEMP,
> // SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG
> //SYSINDD   *
>SORT FIELDS=COPY
>INCLUDE COND=(6,1,BI,EQ,X'11',&,
>45,44,CH,EQ,
>C'deleted.dsn ')
>OPTION VLSHRT
> /*
> //PRINTEXEC PGM=IDCAMS
> //IN1  DD  DISP=(OLD,PASS),DSN=&&TEMP
> //SYSPRINT DD SYSOUT=*
> //SYSINDD   *
>  PRINT INFILE(IN1) DUMP
> /*
>
> Type 17 is pretty easy becaue the dsn is part of the header so you know
the
> offset in each record.  Some other smf record types are not so simple
> because you need to use the triplet for the offset to the section that
had
> the data.  Also, I used a length of 44 for the dsn field.  If your dsn is
> less tan 44, you can shorten the length and get rid of the trailing
blanks.
>
>
> Have a nice day,
> Dave Betten
> DFSORT Development, Performance Lead
> IBM Corporation
> email:  [EMAIL PROTECTED]
> DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
>
> IBM Mainframe Discussion List  wrote on 12/13/2007
> 11:51:04 AM:
>
> > On one of our account, a production dataset is deleted. We
> > Greetings,
> >
> > On one of our account, a production dataset is deleted. We have to
> > find the userid of the person who deleted that specific dataset. I
> > know that we can extract the SMF records with IFASMFDP and Sort the
> > record type 17 for it. However, I am not sure what would be the sort
> > JCL and sort control statements for it.
> >
> > Could you please guide me or provide a sample jcl which could
> > extract the SMF dataset and find the userid.
> >
> > Any help is greatly appreciated.
> >
> > Thanks
> > Vijay
> >
> >
> >   5, 50, 500, 5000 - Store N number of mails in your inbox. Go to
> > http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> > Search the archives at http://bama.ua.edu/archives/ibm-main.html
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Extracting SMF Records to find the userid who deleted a dataset

[David Betten]

You can try this

//SORT  EXEC PGM=SORT,REGION=5000K
//SYSPRINT DD  SYSOUT=*
//SYSOUT   DD  SYSOUT=*
//SORTIN   DD  DISP=SHR,DSN=inputsmf
//SORTOUT  DD  DISP=(NEW,PASS),DSN=&&TEMP,
// SPACE=(CYL,(50,20),RLSE),DCB=*.SORTIN,UNIT=SYSLG
//SYSINDD   *
   SORT FIELDS=COPY
   INCLUDE COND=(6,1,BI,EQ,X'11',&,
   45,44,CH,EQ,
   C'deleted.dsn ')
   OPTION VLSHRT
/*
//PRINTEXEC PGM=IDCAMS
//IN1  DD  DISP=(OLD,PASS),DSN=&&TEMP
//SYSPRINT DD SYSOUT=*
//SYSINDD   *
 PRINT INFILE(IN1) DUMP
/*

Type 17 is pretty easy becaue the dsn is part of the header so you know the
offset in each record.  Some other smf record types are not so simple
because you need to use the triplet for the offset to the section that had
the data.  Also, I used a length of 44 for the dsn field.  If your dsn is
less tan 44, you can shorten the length and get rid of the trailing blanks.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/13/2007
11:51:04 AM:

> On one of our account, a production dataset is deleted. We
> Greetings,
>
> On one of our account, a production dataset is deleted. We have to
> find the userid of the person who deleted that specific dataset. I
> know that we can extract the SMF records with IFASMFDP and Sort the
> record type 17 for it. However, I am not sure what would be the sort
> JCL and sort control statements for it.
>
> Could you please guide me or provide a sample jcl which could
> extract the SMF dataset and find the userid.
>
> Any help is greatly appreciated.
>
> Thanks
> Vijay
>
>
>   5, 50, 500, 5000 - Store N number of mails in your inbox. Go to
> http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: KSDS CA split, no CI split

[David Betten]

Try increasing your index CI size.  Sometimes when the keys are very unique
you get a situation where the index CI can't hold enough pointers for an
entire data CA.  That might be causing the CA splits.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 12/02/2007
06:11:44 PM:

> The following scenario:
> KSDS defined with RECSZ(100 100), FSPC(15 15), CISZ(4096). Records
> loaded, with evenly distributed key values (i.e. start 1, increment 10).
> 2 records.
> Then I inserted 500 evenly distributed records (start=5, increment=100).
>
> Listcat shows no CI splits and 3 CA splits.
>
> Q: how it is possible to have CA split without CI split ?
>
>
> BTW: I increased CA freespace to 30% without effect. Increasing CI
> freespace finally gave me 0 splits at all.
>
> --
> Radoslaw Skorupka
> Lodz, Poland
>
>
> --
> BRE Bank SA
> ul. Senatorska 18
> 00-950 Warszawa
> www.brebank.pl
>
> Sąd Rejonowy dla m. st. Warszawy
> XII Wydział Gospodarczy Krajowego Rejestru Sądowego,
> nr rejestru przedsiębiorców KRS 025237
> NIP: 526-021-50-88
> Według stanu na dzień 01.01.2007 r. kapitał zakładowy BRE Banku SA
> (w całości opłacony) wynosi 118.064.140 zł. W związku z realizacją
> warunkowego podwyższenia kapitału zakładowego, na podstawie uchwał
> XVI WZ z dnia 21.05.2003 r., kapitał zakładowy BRE Banku SA może
> ulec podwyższeniu do kwoty 118.760.528 zł. Akcje w podwyższonym
> kapitale zakładowym będą w całości opłacone.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: How do parallel DFSORTs communicate with each other ?

[David Betten]

Good answer Martin.  The only thing I'd add is that we have an info APAR,
II13495, that provides guidance for setting the DFSORT defaults related to
storage.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 11/13/2007
03:01:07 AM:

> Tobias asked:
>
> > But i now want to know, where does DFSORT get the information how much
> storage is allocated to other DFSORT instances.
>
> Ahead of Dave Betten (of DFSORT Development) answering here's a reply
that
> I hope will be useful to you...
>
> Individual DFSORT instances DON'T communicate with each other. But they
do
> ascertain the state of available memory using the STGTEST SYSEVENT system

> call. Since 1997 - when we introduced "Dynamic Hipersorting" with Release

> 13 - these checks for available memory happen at various stages during
the
> SORTIN phase (while the work files are being built). This "repetitive
> checking" regime reduces the risk of two sorts grabbing "the same piece"
> of memory.
>
> Note: By using STGTEST SYSEVENT DFSORT is actually conditioning its usage

> not just relative to other DFSORT invocations but relative to other
> address spaces and system conditions in general.
>
> In short, even in the same address space (which is one of the DB2 Version

> 8 Utilities scenarios - with Parallel Index Build etc) DFSORT invocations

> don't communicate with each other. But then you don't need them to. :-)
>
> Dave's a great friend and I'm sure he'll give you a better answer. But he

> won't give it to you for another 4 or 5 hours. :-)
>
> Cheers, Martin
>
> Martin Packer
> Performance Consultant
> IBM United Kingdom Ltd
> +44-20-8832-5167
> +44-7802-245-584
> [EMAIL PROTECTED]
>
>
>
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
>
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Performance comparison: Hiperbatch, BLSR, SMB?

[David Betten]

Also, remember that for Hiperbatch only the Data component is kept in the
DLF object.  So you would still have to deal with I/Os to the index.  You
can provide enough buffers to keep the index set in storage but you don't
get lookaside on the sequence set.  With BLSR, you can avoid the seqence
set I/Os too.  So you have to see which is going to provide the greater
benefit, eliminating all of the data I/Os with Hiperbatch, or eliminating
more index I/Os (and hopefully some data I/Os) with BLSR.  Which provides
greater benefit is going to depend on the reference patterns of the jobs
accessing the file (that's a fancy way of saying YMMV!). :-)

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 11/12/2007
07:44:59 AM:

> Alex Tough asked:
>
> > If you have all or most of the cluster allocated to a DLF data
> > object, why would BLSR be faster ?
>
> It might depend on what you mean by "faster"...
>
> Note that nowadays a DLF (or any) hiperspace would be backed by central
> but would still require a more distant data movement. And, in principle,
> VSAM LSR (whether via BLSR or not) ought to be in the job's own address
> space. So, in CPU terms I'd speculate that BLSR might cost less CPU. That

> probably wouldn't turn into elapsed time differences that were measurable

> - but it just MIGHT.
>
> Another thing to remember, though, is that Hiperbatch keeps the memory
> "pro bono publico" whereas BLSR doesn't. The affordability of Hiperbatch
> may be much greater for CONCURRENT jobs. In the same vein with multiple
> BLSR jobs each job would have to populate the buffer pool itself. So
> initially you might get a lot of misses.
>
> All a long-winded way of saying "YMMV". :-)
>
> Cheers, Martin
>
> Martin Packer
> Performance Consultant
> IBM United Kingdom Ltd
> +44-20-8832-5167
> +44-7802-245-584
> [EMAIL PROTECTED]
>
>
>
>
>
>
>
>
>
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU
>
>
>
>
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Performance comparison: Hiperbatch, BLSR, SMB?

[David Betten]

Remember there are two types of DLF objects (Hiperbatch Retain and
non-Retain)

Non-Retain is the one that gets deleted when the open count for the dataset
is 0.  This is the one that was intended for sequential use and you didn't
need a DLR object as large as the dataset.  Just one large enough that
concurrent sequential readers would benefit.  Reader 1 reads from disk and
places copy in Hiperbatch, readers 2, 3, 4, etc. read the Hiperbatch copy.
If the DLF object can hold 100 records than you just hope the other readers
have all gotten to record 1 before the first one gets to record 101.  Sort
of a moving window.

Retain is the one that stays there until you explicility delete it.  This
is good for both sequential and random, especially if it's large enough to
fit the entire dataset.  You load the whole thing into memory once and then
everyone reads the Hiperbatch copy in memory.

Just thought I'd pass on that distinction since it's important when talking
about whether sequential or random can benefit.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 11/09/2007
04:07:56 PM:

> > -Original Message-
> > From: Gerhard Adam [mailto:[EMAIL PROTECTED]
> > Sent: Friday, November 09, 2007 1:57 PM
> > To: IBM-MAIN@BAMA.UA.EDU
> > Subject: Re: Performance comparison: Hiperbatch, BLSR, SMB?
> >
> > Unfortuanately I haven't looked up this stuff in a long time, so I
might
> > be wrong.  But IIRC, Hiperbatch is intended for sequential access and
is
> > counter-productive for random files.  Since it uses a Most Recently
Used
> > algorithm (instead of LRU), the intent was to ensure that the most
recent
> > access to a record was the most eligible for getting discarded from
memory
> > (since this represented the last reader of the data).
>
> Well, I don't know about your memory, but the latest version does indeed
> support random access, AFTER the DLF has been loaded sequentially.
>
> Don't know what algorithm it uses, but my SMF gurus tell me they can
prove
> it is being used, and in fact the batch jobs run longer if anything
happens
> to Hiperbatch, so I think it's working.  Our access is almost all random.
>
> > The whole point was to avoid having records discarded because of age
just
> > ahead of someone that was reading the file sequentially.
> >
> > Also, another point was that the I/O counts were unaffected since the
> > application was unaware that it was using Hiperbatch, so that
information
> > is largely irrelevant.
>
> I didn't know that, but it makes sense.  Thanks.
>
> > Anyway ... here's hoping my memory isn't completely gone
>
> Not completely.  :)
>
> Peter
>
> This message and any attachments are intended only for the use of
> the addressee and
> may contain information that is privileged and confidential. If the
> reader of the
> message is not the intended recipient or an authorized representative of
the
> intended recipient, you are hereby notified that any dissemination of
this
> communication is strictly prohibited. If you have received this
> communication in
> error, please notify us immediately by e-mail and delete the message and
any
> attachments from your system.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Each CPU usage

[David Betten]

I think the debate of whether this story is plausible or not is better
taken offline.  Or you can just agree to disagree.  I think the important
point that everyone would agree on is that when you are reaching maximum
CPU capacity, it's always a good idea to gather data and understand what's
driving that CPU before you upgrade.  You may find tuning opportunities to
reduce the cpu requirements, or you may just get a better understanding of
what workloads are driving that usage and what kind of latend demand you
have.  100% CPU might not be so bad if your critical work is getting what
it needs and discretionary work is taking the rest (I know the decision on
what's discretionary can be a political battle!).

CICS performance statistics (SMF 110) have all kinds of great information
to help you understand the characteristics of your transaction workload
including CPU usage.  I'm no CICS expert but I've seen that data put to
great use by those who know how to use it.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 08/29/2007
08:55:47 AM:

> >It was a custom application written for a very large customer.
> Someone else mentioned >that it seemed an improbable scenario. Be
> that as it may, it happened.
>
> The problem isn't simply with the scenario of acquiring a lock, but
> rather than there was an apparent upgrade of 250% (regarding the
> number of engines) and apparently NOBODY bothered to gather data to
> determine the basis for such a change.
>
> Secondly, your assertion that the processor was at 100% utilization,
> suggests that the four engines were occupied in spin loops, which
> also suggests that no other work was capable of running (which is
> another implausible scenario).
>
> Since the conclusion was that they didn't even require a 4-way after
> the problem was "solved", it is clear that the utilization of the
> processor was driven by the spin loops which leads one to conclude
> that there could have been no other running work on the system
> beyond four transactions.  As I indicated, this stretches the bounds
> of even anectodal evidence and would require a much more detailed
> explanation to approach plausibility.
>
> Adam
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: What's the new support for large files called?

[David Betten]

Check out this manual for a good description of large format datasets.

z/OS
DFSMS: Using the New Functions
Document Number SC26-7473-02

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 08/03/2007
01:46:12 PM:

> > -Original Message-
> > From: IBM Mainframe Discussion List
> > [mailto:[EMAIL PROTECTED] On Behalf Of Lindy Mayfield
> > Sent: Friday, August 03, 2007 12:40 PM
> > To: IBM-MAIN@BAMA.UA.EDU
> > Subject: What's the new support for large files called?
> >
> >
> > When you forget the terminology it's impossible to Google or anything.
> > Sorry about the stupid question, but:
> >
> > What is this new support for large datasets in, I think z/OS
> > 1.7 or 1.8
> > called?  I wanted to Google to find the size differences.
> >
> > I've Googled for 10 minutes and without the proper keywords it isn't
> > easy. (-:
> >
> > Thanks!
> > Lindy
>
> Do you LBI? That's for LRECL > 32K. Or "Large Dataset" (extended format)
> for datasets with >65K tracks on a single volume.
>
> --
> John McKown
> Senior Systems Programmer
> HealthMarkets
> Keeping the Promise of Affordable Coverage
> Administrative Services Group
> Information Technology
>
> The information contained in this e-mail message may be privileged
> and/or confidential.  It is for intended addressee(s) only.  If you are
> not the intended recipient, you are hereby notified that any disclosure,
> reproduction, distribution or other use of this communication is
> strictly prohibited and could, in certain circumstances, be a criminal
> offense.  If you have received this e-mail in error, please notify the
> sender by reply and delete this message without copying or disclosing
> it.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SOC4 in DFSORT with DB2

[David Betten]

I don't want to circumvent the PMR process but thought I'd offer some a
thought based on this information.  Double check the storage class that the
sortwork datasets are being assigned to and make sure the default unit
count is 1.  Sortwork datasets cannot be multi-volume so there's no point
in having a default greater than one.  If the default volume count is
something high, like 59, then you're probably going to exceed the TIOT
threshold.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]

DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 08/01/2007
11:30:01 AM:

> George,
>
> I should have mentioned that our Abend0C4 was in IEFAB42A, or IEFAB421,
> depending on what you are looking at.
>
> John
>
>
> Confidentiality notice: The information included in this e-mail,
> including any attachments, is for the sole use of the intended recipient
> and may contain confidential and privileged information. Any
> unauthorized review, use, disclosure, distribution or similar action is
> prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies of the original message immediately.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of Tergerson, John
> Sent: Wednesday, August 01, 2007 11:21 AM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Re: SOC4 in DFSORT with DB2
>
> George,
>
> We got an 0C4, and also a U0083 symptom using DFSORT to do reorgs on a
> tablespace with a lot of indexes.
>
> The DBA had set up sortnum to 160 so they would never have to compute
> sort work space.  When they reduced it to 80, things worked fine.  The
> tablespace with the problem was small, but has lots of indexes.  Large
> tablespaces with fewer indexes could be reorged with larger sortnum.
>
> There are two new APARs we caused.  Look at OA21854 and  darn, I
> can't get to IBMLink to find the other, if it was even assigned yet.
>
> In our case, the large number of sortnum caused a condition that
> exceeded the allowable size of the TIOT.  The allocation APAR is to give
> the user a better symptom than abend0C4 to tell the user what is wrong.
> The 0C4 is because the slot allocation was trying to free up was for
> VIO, which does not have a control block they were looking for.
>
> The DB2 APAR is to address an issue of why they were asking for an
> enormous number of sortwork datasets.  I'll post that whenever I can
> find it.  Asking for sortnum of 80 for most reorgs, though, gave us a
> welcome bypass.
>
> Hope that helps.
>
> John
>
>
> Confidentiality notice: The information included in this e-mail,
> including any attachments, is for the sole use of the intended recipient
> and may contain confidential and privileged information. Any
> unauthorized review, use, disclosure, distribution or similar action is
> prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies of the original message immediately.
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> Behalf Of George Bly
> Sent: Wednesday, August 01, 2007 8:22 AM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: SOC4 in DFSORT with DB2
>
> HI All
> We went to DB2V8 this week.  Now we are getting S0C4's in DFSORT when
> using
> DB2 utilities.
> I've read the informational APAR and I have it in correctly after my
> SyncSort libs.
> I am working with IBM but has anyone else run into this problem.
> By the way:  IBMLINK went down when we first started researching the
> problem.  It was very inconvenient to say the least.
> Thanks!!
> George
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send
> email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO Search
> the archives at http://bama.ua.edu/archives/ibm-main.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT V1R5 - ICE751I and ICE752I

[David Betten]

If your system is at z/OS 1.7 or higher, you can exceed 4000 cylinders on a
single volume by adding DSNTYPE=LARGE to your sortwork DDs and they will be
allocated as large format datasets.  If you don't code JCL sortworks and
let DFSORT dynamically allocate the required sortwork space ( our
recommended method), the sort work datasets are automatically allocated as
large format.

For your other points, I'll take those under advisement with my team.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/01/2007
10:38:55 PM:

> Hi,
>
> Thanks everybody for the feedback.
> Somebody else was working on the problem today because I actually tried
> to get some sleep.
>
> So I forwarded all these postings to him and  Dave , we would sent you
> the FULL output.
>
> Summary:
>
> I really would suggest that we make "IBM Software" more user friendly :
>
> a) Why not print a warning message to the User indicating that any
> SORTWK99 dataset will only use ONE volume
> b) Why not print a message to the user stating that the Memory
> allocation is too small.
> c) Why do we still have a limit in zOS with regards the Primary Cylinder
> allocation. Apparently it's 4000 Cylinders and I found this out the hard
> way last night.
> d) When abending, please tell me more. Currently you print messages that
> says "For the IBM support rep." I thought IBM stopped doing this in
> the 80's
>
> You do print how many records are about to be sorted, so you are half
> way there ( Don't take me halfway... I want it all   and I want it
now )
>
> Anton
>
> _David Betten wrote:
> > I already answered Anton in another post but I wanted to respond to
this
> > and make clear that DFSORT work files cannot span multiple volumes.  If
you
> > coded UNIT=(SYSDA,10) on a SORTWKnn DD statement, DFSORT would still
use
> > only 1 volume for each SORTWK data set.
> >
> > Have a nice day,
> > Dave Betten
> > DFSORT Development, Performance Lead
> > IBM Corporation
> > email:  [EMAIL PROTECTED]
> > 1-240-715-4655, tie line 268-1499
> > DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
> >
> > IBM Mainframe Discussion List  wrote on
07/01/2007
> > 02:10:23 PM:
> >
> >>> If necessary and the file is SMS managed add a volume parameter to
> >> the UNIT parameter, eg. UNIT=(SYSDA,10).
> >>
> >> It doesn't have to be SMS managed for the volume parameter to work.
> >> -
> >> Too busy driving to stop for gas!
> >>
> >> --
> >> For IBM-MAIN subscribe / signoff / archive access instructions,
> >> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> >> Search the archives at http://bama.ua.edu/archives/ibm-main.html
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> > Search the archives at http://bama.ua.edu/archives/ibm-main.html
> >
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT V1R5 - ICE751I and ICE752I

[David Betten]

I already answered Anton in another post but I wanted to respond to this
and make clear that DFSORT work files cannot span multiple volumes.  If you
coded UNIT=(SYSDA,10) on a SORTWKnn DD statement, DFSORT would still use
only 1 volume for each SORTWK data set.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/01/2007
02:10:23 PM:

> >If necessary and the file is SMS managed add a volume parameter to
> the UNIT parameter, eg. UNIT=(SYSDA,10).
>
> It doesn't have to be SMS managed for the volume parameter to work.
> -
> Too busy driving to stop for gas!
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT V1R5 - ICE751I and ICE752I

[David Betten]

Anton,
  If you send the entire sysout directly to my email
([EMAIL PROTECTED]) I can probably diagnose your abend easier.But at
first glance, I suspect the very limited main storage you are allowing the
sort (524288) is causing an intermediate merge and that's when you are
running out of sortwork space.   But again, I'd need to see the entire
sysout to be sure.

Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/01/2007
12:38:15 PM:

> Yes... thanks for all the feedback and my initial posting was more out
> of frustration with DFSORT and SMS.
>
> So here comes more detail :
>
> The Scene of the crime :
>
> Input Dataset contains 4,826,605 records of 2000 bytes each.
>
> The Sort Control Cards :
>
> ICE143I 0 BLOCKSET SORT  TECHNIQUE SELECTED
> ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS,
> EXAMPLES AN
> ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 03:58 ON
> SUN JUL
>  SORT FIELDS=(49,6,A,59,8,A,91,6,A,99,8,A,81,2,A,
>   97,2,A,1947,3,A,55,4,A,1950,2,A),FORMAT=CH
> ICE201I E RECORD TYPE IS F - DATA STARTS IN POSITION 1
> ICE751I 0 C5-K90007 C6-K90007 C7-K9 C8-K90007 E4-K90007 C9-BASE
> E5-K14794
> ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE
SELECTED
> ICE088I 1 HWLK221A.STEP220 .SORT, INPUT LRECL = 2000, BLKSIZE =
> 4000, TYPE =
> ICE092I 0 MAIN STORAGE = (350,524288,524288)
> ICE156I 0 MAIN STORAGE ABOVE 16MB = (458304,458304)
> ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0
> ,SPANINC=RC16,VLSCMP=N,SZERO=Y,
> ICE128I 0 OPTIONS:
> SIZE=524288,MAXLIM=2097152,MINLIM=524288,EQUALS=Y,LIST=Y,ERET
> ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO
> ,WRKSEC=Y,OUTSEC=Y,VERIFY=Y,CHALT=
> ICE130I 0 OPTIONS: RESALL=0,RESINV=0,SVC=109
> ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,S
> ICE131I 0 OPTIONS:
> TMAXLIM=5242880,ARESALL=0,ARESINV=0,OVERRGN=0,CINV=Y,CFW=Y,DS
> ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE
>,EXITC
> ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX
> ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMI
> ICE235I 0 OPTIONS: NULLOUT=RC0
> ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
> ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
> ICE750I 0 DC 0 TC 9653212000 CS DSVVV KSZ 45 VSZ 45
>
> Sort Step JCL :
>
> XXSORT PROC PROG=ICEMAN,
> XXWTR1='*',
> XXCYL=15,
> XXDEVTYP=TMPDA,
> XXRGIN=6M
> XXSORT EXEC PGM=&PROG,REGION=&RGIN,PARM='SIZE=350'
> IEFC653I SUBSTITUTION JCL - PGM=ICEMAN,REGION=6M,PARM='SIZE=350'
> XXSYSOUT   DD  SYSOUT=&WTR1
> IEFC653I SUBSTITUTION JCL - SYSOUT=*
> XXSYSOUX   DD  SYSOUT=&WTR1
> IEFC653I SUBSTITUTION JCL - SYSOUT=*
> XXSYSLMOD  DD  SPACE=(3600,(20,20,1)),DSN=&&GOSET,
> XX UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(3600,(20,20,1)),DSN=&&GOSET,UNIT=TMPDA
> XXSYSLIN   DD  SPACE=(80,(10,10,)),DSN=&&LOADSET,
> XX UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(80,(10,10,)),DSN=&&LOADSET,UNIT=TMPDA
> XXSYSUT1   DD  SPACE=(1000,(60,20)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(1000,(60,20)),UNIT=TMPDA
> //SORTWK01 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> //SORTWK02 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK02 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SORTWK03 DD SPACE=(CYL,(4000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK03 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SORTWK04 DD SPACE=(CYL,(2000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK04 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SORTWK05 DD SPACE=(CYL,(3000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK05 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SORTWK06 DD SPACE=(CYL,(3000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK06 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SORTWK07 DD SPACE=(CYL,(1000,500)),UNIT=TMPDA,
> //DCB=(LRECL=2000,BLKSIZE=28000,RECFM=FB)
> X/SORTWK07 DD  SPACE=(CYL,(&CYL,&CYL)),UNIT=&DEVTYP
> //*
> IEFC653I SUBSTITUTION JCL - SPACE=(CYL,(750,750)),UNIT=TMPDA
> //SYSPRINT DD SYSOUT=*
> //SORTIN   DD DSN=HWL.HWLK228P.ARCHIVE,DISP=OLD
> //SORTOUT  DD DSN=HWL.HWLKSORT.HWLK228P,
> //DISP=(NEW,PASS,DELETE),
> //STORCLAS=LRGETAPE,UNIT=3490,
> //DCB=(HWG.PATTERN.RECL2000,BLKSIZE=0)
> //SYSINDD *
> //*
>
> The WORK packs available :
>

Re: DFSORT V1R5 - ICE751I and ICE752I

[David Betten]

Anton,
  Lizette is correct that the diagnostic messages are for use by our
L2/L3 teams and that's why they are not documented.  The ICE751I and
ICE752I messages are informational (note they end with an I) and are issued
for both successful and unsuccessful sorts.  Usually when a sort fails,
there is an  ICExxxA message to indicate the cause.  If you would like to
send me the sysout of your failed sort, I'd be happy to look at it and help
determine the cause of your abend.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 07/01/2007
11:27:46 AM:

> Anton,
>
> I am not sure what is your issue.  Not all IBM messages are documented
> because some are for internal diagnostics by the IBM L2/L3 teams (if
> needed).  So I would not expect every message documented because it would
be
> meaningless to me since I do not have access to the IBM Code for DFSORT.
>
> It seems to me you are upset over the fact that the job ran for 2 hours
> before abending.  That maybe a valid issue.  Since you did not post any
> specifics (SORT CONTROL CARDS, type of data being sorted or how much data
> being sorted) it is hard to tell if the sort needs tuning.
>
> If you turned the DIAGs on for this job, then you probabaly need to open
an
> ETR with IBM to resolve the B37 issue.
>
> Lizette
>
>
> > Hi,
> >
> > I abended twice this weekend on SB37-04, after running for TWO hours
and
> > when I look at the output of the DFSORT , I see ICE751I and ICE752I
> > messages.
> >
> > The IBM manual says :
> >
> > Explanation: DFSORT diagnostic message for use by IBM software support
> > personnel.
> >
> > Conclusion :
> >
> > That sounds line "s'mee" but it still does not mean much.
> >
> > Can the IBM DFSORT team not hire a H1B visa person for $8 and hour and
> > document all these possible output messages ?
> >
> > Anton
> >
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: RESTART SINGLE JOB STEP

[David Betten]

 I THINK (not positive) that if you added COND=(0,LE) to the job statement,
that would force all the steps after the restart step to flush.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 04/10/2007
12:50:05 PM:

> >
> > RESTART=stepnameor step+proc
> However, that restarts at the given step but continues to execute the
> rest of the steps following it.  I think she wanted to execute just one
> step in the middle of the job.
>
> --
> Bruce A. Black
> Senior Software Developer for FDR
> Innovation Data Processing 973-890-7300
> personal: [EMAIL PROTECTED]
> sales info: [EMAIL PROTECTED]
> tech support: [EMAIL PROTECTED]
> web: www.innovationdp.fdr.com
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Smartbatch for OS/390 vs Batchpipes OS/390

[David Betten]

I don't remember all the specifics but initially IBM offered Batchpipes as
a standalone product.  It was later merged into the Smartbatch product
which was a combination of Batchpipes with some automatic buffering and
parallel processing tools.  Then after a few years, Smartbatch was
withdrawn and but the Batchpipes component was again offered by itself.   I
believe Batchpipes is still supported.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/31/2007
08:41:40 AM:

> I also find Batchpipes V2 R1 as the only release for Version 2.
> Program number is 5655-D45. Available May 2000.
>
> I find nothing later.
>
> For some non-IBM information see:
> http://www.jpaulmorrison.com/cgi-bin/wiki.pl?BatchPipes
>
>
> Birger Heede 
> IBM Denmark
>
> oktg wrote:
> > Hi,
> >
> > In an earlier posting I asked if somebody could tell me the latest
> > available release of Smartbatch for OS/90. I have now found an IBM
> > PAL which says:
> >
> > Programming Announcement letter ZP01-0183 dated March 06, 2001
> > Versions to be withdrawn from service April 04, 2004:
> > Product Name   Version  Product
> >  Number
> >
> > IBM SmartBatch for OS/390  15655-A17
> >
> > So Smartbatch is discontinuedbut now my want to know the latest
> > available releasenr of BatchPipes OS/390 ???
> >
> > On IBM booksshelves i found documents for:
> >
> >  IBM BatchPipes OS/390 V2R1
> >
> > Many thanks in advance
> >
> > Frank Silven
> >
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSort oddity

[David Betten]

Neal Eckhardt wrote:
> I have been testing different values for HIPRMAX to limit the amount
> of expanded storage it uses.
>
> It appears that DFSort uses a Hiperspace OR a Data Space. Specifying
> HIPRMAX=1% will result in no usage of Data Spaces. Why not?
>
> The EXCP counts on SORTIN and SORTOUT are different depending on
> wether a Hiperspace is used or a Data Space is used (counts below).
> Why would SORTIN and SORTOUT EXCP counts be different?
>
> Neal
>
> Dataspace:
>
> DEVICE ACTIVITY:  (BLOCK COUNT, DDNAME, DEVICE, DEVICE TYPE)
> 229 SORTIN   1123 3390125 SORTOUT  1228 3390
> 193 SORTWK04 103E 339
> 195 SORTWK02 103F 3390191 SORTWK01 1227 3390
> 184 SORTWK03 1229 339
>   2 SORTWK05 1228 3390  1 SORTWK06 1228 3390
>
> Hiperspace:
>
> DEVICE ACTIVITY:  (BLOCK COUNT, DDNAME, DEVICE, DEVICE TYPE)
> 117 SORTIN   1123 3390299 SORTOUT  1227 3390
> 346 SORTWK02 103E 3390
> 332 SORTWK05 103E 3390340 SORTWK03 103F 3390
> 321 SORTWK01 1228 3390
> 321 SORTWK04 1229 3390  1 SORTWK06 1227 3390


You are correct that DFSORT will use Hiperspace OR Dataspace.  It will not
use them together.  Note that in DFSORT V1R5 it's Memory Object or
Hiperspace or Dataspace.

When you specify HIPRMAX=1% and there is no usage of Dataspace, does the
sort use Hiperspace instead or neither?

The EXCP counts are going to vary between the two methods because of the
way DFSORT allocates storage for i/o buffers.  DFSORT allocates different
control blocks and internal storage areas differently for the two methods.

If you would like to investigate this in more detail, you can send me the
sysouts from your runs and I can take a look at them.

It would be helpful to understand what your objective is in this testing.
Generally, we recommend you let DFSORT select the method that's optimal for
each sort based on the characteristics of the sort and the available
resources on the system.

Dave Betten
IBM DFSORT Performance Lead
[EMAIL PROTECTED]

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: ACS Routine works for JCL but not in ISPF 3.2

[David Betten]

Does the data set get assigned to the same storage class when allocated via
JCL and ISPF 3.2? Storage class is assigned prior to storage group and your
storage group routine could be assigning based on the storage class.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 01/02/2007
02:32:58 PM:

> > -Original Message-
> > From: Vinson Lee [mailto:[EMAIL PROTECTED]
> > Sent: Tuesday, January 02, 2007 1:25 PM
> > To: 'IBM Mainframe Discussion List'
> > Cc: McKown, John
> > Subject: RE: ACS Routine works for JCL but not in ISPF 3.2
> >
> >
> > I don't see a reference to XMODE in any of our ACS routines.
> > Is there maybe
> > a default?
> >
> > Thanks.
> >
> > Vince
>
> XMODE means execution mode. Its value is set depending on the
> environment during the execution of the ACS routine. However, if you
> don't reference it in your ACS routine, it does not used for anything.
>
> Likely you will need to post your ACS routine, otherwise we are all
> flying blind.
>
> --
> John McKown
> Senior Systems Programmer
> HealthMarkets
> Keeping the Promise of Affordable Coverage
> Administrative Services Group
> Information Technology
>
> The information contained in this e-mail message may be privileged
> and/or confidential.  It is for intended addressee(s) only.  If you are
> not the intended recipient, you are hereby notified that any disclosure,
> reproduction, distribution or other use of this communication is
> strictly prohibited and could, in certain circumstances, be a criminal
> offense.  If you have received this e-mail in error, please notify the
> sender by reply and delete this message without copying or disclosing
> it.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT SORTWK questions

[David Betten]

See my answers below:


>We had a very large DFSort job that abended because of insufficient
>Sortwk space with message ICE083A, indicating that 4 times 65535 tracks
>were not enough.

I'm guessing your system is not a z/OS 1.7 yet.  Just an fyi that for
systems at 1.7 or later, DFSORT allocates the sortworks as large format
so they can exceed 65535 tracks (providing there are volumes with that
much space available).

>Looking for options to raise the default of 4 Sortwk's, I ran into 2
>questions to which I could not find the answer:



>1.  I cannot find any recommandations in the DFSort manuals about
>the optimum number of Sortwk's. Spreading more Sortwk's about more
>devices is in generally advisable, but in modern storage devices less
>relevant than before.
>Does DFSort really not care whether it has 5, 50 or 255 Sortwk's and how
>do I determine the optimum number of Sortwk's?

There's not much performance difference when you go up to say 16 or 32.
However, you don't want to do something drastic like 255.  The reason is
that it can cause the sortworks to become fragmented and then the merge
of all those strings can take longer.  It's difficult to proclaim one
specific number as optimal because factors like I/O contention in your
environment can influence what performs best.

>2.  From the manual I understood that the number of Sortwk's that I
>specify in the DYNALOC parameter of the ICEMAC macro's specified the
>*maximum* number of Sortwk's DFSort was allowed to take and that it does
>not force dymanic allocation, nor the number of Sortwk's. Tests we did
>seem to show that DFSort, when it dynamically allocates Sortwk's, will
>always allocate the number of Sortwk's in the ICEMAC macro. Is this
>correct?

DFSORT will almost always allocate the number of sortworks you specify in
the DYNALOC (installation default) or DYNALLOC (runtime specified).
However, if you have JCL sortworks coded but DYNAUTO=IGNWKDD
is not in effect, then it will use the JCL sortworks and not perform
dynamic allocation.  When dynamic allocation is used, the sortwork space
required is spread across all the sortworks.  So more sortworks does not
mean
more space, but instead it means more/smaller sortwork datasets.

I'm sorry if our manuals are unclear and will see if we can revise to make
it more clear.  I'll add that to the list of things to consider
for future releases.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DFSORT and SS

[David Betten]

Frank Yaeger is always the best at answering these types of questions but
he's on vacation this week.  So I forwarded your question to Vicky Vezinaw
and she supplied the following answers.

ANSWER to Q1:

When you say:
 I used the following syntax:

 OMIT COND=(5,32000,SS,EQ,C'SMITH')
 OPTION COPY,VLSHRT

 Unfortunately it doesn't work.

Did you get a syntax error and are you using DFSORT R14 with the above
statements?  If not please be more specific about the problem you are
encountering.  If you are getting a syntax error, then you need PTF UQ90053
applied.  In that PTF, the maximum length for an SS field used with OUTFIL
INCLUDE and OUTFIL OMIT has been raised to 32752.

ANSWER to Q2:
-
You can run a simple job to get the minimum and maximum record length.  I
derived this JCL from an example in the DFSORT Application Programming
Guide (DISPLAY Operator example)

   DISPLAY FROM(INV) LIST(RDWLIST1) -
   TITLE('No Frills RDW Report') -
   ON(NUM) -
   ON(VLEN) -
   ON(1,4,HEX) -
   MINIMUM('Smallest') -
   MAXIMUM('Largest')

The INV is the DD for the input data set.  RDWLIST1 is the DD for the
output report data set.  The rdw's will be printed with the smallest and
largest RDW printed at the end of the report.


ANSWER to Q3:
-
You can use OUTFIL to accomplish this task.  I also derived the following
JCL from an example in our DFSORT Application Programming Guide (OUTFIL
description, under the SAVE description)

   OUTFIL INCLUDE=(8,6,CH,EQ,C'ACCTNG'),FNAMES=GP1
   OUTFIL SAVE,FNAMES=NOTINC

The DD GP1 data set will have the included records.  The DD NOTINC data set
will have all of the other records (in other words the records that did not
meet the INCLUDE condition).

I hope this helps you.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
1-240-715-4655, tie line 268-1499
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 08/01/2006
12:41:43 PM:

> I have variable length records to filter.
> I should omit all records containing given string, i.e. SMITH.
> I used the following syntax:
>
> OMIT COND=(5,32000,SS,EQ,C'SMITH')
> OPTION COPY,VLSHRT
>
> Unfortunately it doesn't work. When I changed the lenght:
> OMIT COND=(5,900,SS,EQ,C'SMITH')
> then some records were filtered. I suspect that only records longer than
> 904 bytes and containing SMITH in first 900 bytes (excluding RDW).
>
> Q: How to fix it, I mean omit all records containing SMITH, regardless
> of the length ?
>
>
> Q2: What is simple method to find out records length, one by one, or
> just min and max ?
>
> Q3: How can I split output, put record without the string in one
> dataset, and the rest (with the string) in another one ? Should I use
> OUTFIL or rather something else ?
>
> Regards
> --
> Radoslaw Skorupka
> Lodz, Poland
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: What this RMF situation means ????

[David Betten]

You might want to look at the partition data (Overview Option 1 from main
menu followed by CPC Option 3 on next menu) to make sure that the CEC isn't
constrained.  Your LPAR may appear only 60% busy but that might be because
that's as high as it can get because the CEC is constrained.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 06/28/2006
12:26:05 PM:

> Javier,
>
> Looks like you've got a program running and getting pretty good service.
> Workflow percent is 80%?  It's getting 80% of everything it's asking
> for.  That's pretty good.
>
> I'd not put too much stock in the "primary delay" when it targets a
> particular ASID for CPU contention.  It only points to the ASID that is
> using the most CPU, which may or may not be a problem.
>
> You also have to look around the rest of your system, take a sanity
> check, and see if the primary reason is also a viable suspect.  In your
> case, I'd say that it probably isn't, and if you're concerned that your
> IDMSPROD isn't getting the service it needs, you need to dig deeper into
> it using other RMF views or some other tool.
>
> But with an average CPU utilization of 60% for the entire system, I'd
> say you have nothing to worry about.  As we say, it's a "non-issue".
>
> Best regards,
>
> Gary Diehl
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED]
> Behalf Of javier sotela
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: SMF help needed please

[David Betten]

Those don't look like SORT messages but rather something that would come
out of the RMF post processor or exits.  Does your job have an EXITLIB or
STEPLIB that's pointing to downlevel version of the code?  Was there a
message id at the beginning of those messages like ERB that would
indicate who is issuing them.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 06/26/2006
06:35:05 AM:

> Folks,
>
> We recently upgraded to z/OS V1.7 from 1.6 and the JCL that I used under
> version 1.6 to sort the SMF tape before generating SMF reports no longer
> works.
>
> I get the following SMF message:
>
> RECORDS WITH VERSION 717 WERE FOUND IN THE INPUT DATA SET.
> THE EXPECTED VERSION NUMBER IS 715
> RECORDS WITH VERSION NUMBER 7.1.7-74 WERE FOUND IN THE
> INPUT DATASET.  THE EXPECTED VERSION NUMBER IS
> 7.1.5-71.
>
> My old sort parms were:
>
>  //SYSIN   DD *
>   INCLUDE COND=(06,1,BI,GE,X'46',AND,06,1,BI,LE,X'4F')
>   SORT FIELDS=(11,4,CH,A,7,4,CH,A),EQUALS
>   MODS E15=(ERBPPE15,500,,N),E35=(ERBPPE35,500,,N)
>   END
> /*
>
> Does anyone have the new sort parms for version 1.7 that they would be
> willing to share ??
>
> Or if it is not the sort parms, be able to tell me what is wrong with my
> JCL?
>
> Thanks in advance.
>
>
> Richard W. Evans
> Technical Specialist
> Worldspan
> Worldwide Technology Planning & Implementation Department
> System Services Group
> z/OS (MVS) Capacity/Performance
> E-MAIL :  [EMAIL PROTECTED]
>
> Worldspan e-Pricing ... The next Generation in fare search technology -
> http://www.epricing.net
>
> NOTICE:  This e-mail message and all attachments transmitted with it
> contain confidential information intended solely for the use of the
> addressee.  The addressee agrees to keep the information as confidential.
> Any improper use of the information, or dissemination or distribution to
> third parties, is strictly prohibited.  If the reader of this message is
> not the intended recipient, you are hereby notified that any reading,
> dissemination, distribution, copying, or other use of this message or its
> attachments is strictly prohibited.  If you have received this message in
> error, please notify the sender immediately and delete this message and
all
> copies and backups thereof.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: z/OS DASD Allocation using Esoterics

[David Betten]

You may want to look at the DFSMSdfp Storage Administration Reference topic
6.6, SMS Volume Selection for Data Set Allocation.  This discusses in
detail the criteria used by DFSMS for volume selection.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 06/15/2006
03:17:21 PM:

> Raymond's point is excellent.  It is also possible that there was not
> adequate space or there was some other constraint at the time SMS made
the
> decisions.  It would be interesting to see if you got the exact same
> results at different periods, repeating the same experiment.
>
> I think I'm in trouble here because I just implied that the software can
> actually THINK!  I must need a vacation.
>
> Regards,
>
> Rob Weiss
> z/SWITA and z/Series I/T Security and Privacy Consultant
> IBM Software Group Sales
>
> IBM Mainframe Discussion List  wrote on 06/15/2006
> 01:01:40 PM:
>
> > Raymond,
> >
> > Are all of the SMS managed volumes in ENABLE status? Maybe one of the
> > volumes
> > is in QUIESCE status or even DISABLE or DISABLE, NEW.
> > Regards,
> > John
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> > Search the archives at http://bama.ua.edu/archives/ibm-main.html
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Dcollect

[David Betten]

You can download a sample for processing DCOLLECT with DSFSORT from the
DFSORT web site.


http://www-03.ibm.com/servers/storage/support/software/sort/mvs/srtmdwn.html



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 04/04/2006
03:25:56 AM:

> There are quite a few DCOLLECT processing examples on the CBT Tape -
> www.cbttape.org
>
> Download File #1 and have a look.
>
>
> Rob Scott
> Rocket Software, Inc
> 275 Grove Street
> Newton, MA 02466
> 617-614-2305
> [EMAIL PROTECTED]
> http://www.rs.com/portfolio/mxi/
>
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: DB2/DFSORT/MOSIZE/IEFUSI Super Sorting above the bar with DFSORT

[David Betten]

I need to comment on a few items here.

First, I'm pretty sure that the MEMLIMIT in SMFPRMxx is a default that's
used if you don't have MEMLIMIT on your JCL or REGION=0M. I don't believe
it overrides it. So if you really want to put a hard limit on the amount of
above the bar storage someone can get, you need to put it in the IEFUSI. I
know there's a good flow chart on this, I just need to find it again.

Second, if DFSORT can't use Memory Object, it can still use your central
storage via Hiperspace or Dataspace so you may want to look at your HIPRMAX
and DSPSIZE parameters in ICEMAC as well.

Third, generally we recommend customers use the shipped defaults of
MOSIZE=MAX, HIPRMAX=OPTIMAL and DSPSIZE=MAX. DFSORT is designed to look at
available resources and not over commit. But if you don't have a paging
subsystem configured to really back a 30GB central, then I agree you need
to limit it. You might want to consider leaving MOSIZE=MAX and
HIPRMAX=OPTIMAL and setting EXPMAX to a value you are comfortable with.
EXPMAX sets the limit of central storage (in 64-bit env) that can be used
by ALL Memory Object and Hiperspace sorts in the system. You should also
look at EXPOLD which says how much "old" storage is available. I suspect
the paging you saw was because DFSORT went after old pages from long
running address spaces like your online subsystems and that caused them to
page out. You could do something like leave EXPMAX=MAX and change to
EXPOLD=10%. This will allow DFSORT to obtain any "available" pages but
limit page steals from any other address spaces.

Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 03/21/2006
03:53:04 PM:

> Hi,
>
> We have a few exploiters of storage above the bar and mostly it has been
> all good but I have seen some interesting things with DFSORT recently.
>
> DFSORT using the default MOSIZE=MAX used 12G for an index rebuild called
> by DB2 utility PGM=DSNUTILB,REGION=0K on an LPAR with 30GB  running the
> AFQ down to 18326 and inducing some paging on a system where I would not
> have expected to see it.  I specify MEMLIMIT(4G) on most of our systems
> in SMFPRMxx.  Currently since we use DFSORT only for DB2 V8 I had not
> prompted anyone to customize ICEMAC.
>
> I recently added an IEFUSI exit to the mix here but don't yet have any
> code in there to handle MEMLIMIT.
>
> I am considering adjusting both with my current thinking to change
> IEFUSI to honor the specification from SMFPRMxx so only
> GRS/DB2/reallycoolauthorizedguys will override that and to customize
> ICEMAC to change MOSIZE=MAX to MOSIZE=10% to prevent any single DFSORT
> job from getting too giddy when it sees a large AFQ.
>
> What are you doing here?  Anyone else customize MOSIZE in ICEMAC?
> Anything else in the ICEMAC defaults to watch out for?  We have been a
> long time SYNCSORT shop so have been taking a wait and see approach with
> DFSORT tuning.
>
> Best Regards,
>
> Sam Knutson, GEICO
> Performance and Availability Management
> mailto:[EMAIL PROTECTED]
> (office)  301.986.3574
>
> Before you criticize someone, you should walk a mile in their shoes.
> That way, when you criticize them, you're a mile away and you have their
> shoes.
>
> ][
>
> 
> This email/fax message is for the sole use of the intended
> recipient(s) and may contain confidential and privileged information.
> Any unauthorized review, use, disclosure or distribution of this
> email/fax is prohibited. If you are not the intended recipient, please
> destroy all paper and electronic copies of the original message.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: CFW and DFSORT

[David Betten]

I apologize, my earlier append was not clear. DFSORT does check and if CFW
is not active in the controller, then the CFW bit is not turned on. Also, I
would agree that there is a performance benefit of using CFW since you
eliminate the need to write to the NVS. But I have seen many customers
running effectively with CFW=NO. So if you're looking for the high
availability of a GDPS environment and Hyperswap, I don't think disabling
CFW is going to be a show stopper.


Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List  wrote on 02/24/2006
12:16:15 AM:

> Bruce,
>
> Yes I misread that.
>
> But with CFW ON in DFSORT and OFF in the Controller I don't think SORT
turns
> on the CFW bit. I think CFW ON in DFSORT causes a check for CFW ON in the
> controller, and then CFW is set appropriately.
>
> David's reply suggests that SORT sets the CFW bit even when the
controller
> has CFW OFF. The IO chaining behaviour suggests otherwise.
>
> Ron
>
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:[EMAIL PROTECTED] On
> > Behalf Of Bruce Black
> > Sent: Friday, 24 February 2006 10:52 AM
> > To: IBM-MAIN@BAMA.UA.EDU
> > Subject: Re: CFW and DFSORT
> >
> > >
> > > You're from IBM right? So you are saying that ESS and DS8K ignore CFW
> > and
> > > put a copy in the NVS anyway? Then why does DFSORT change its IO
> > chaining
> > > behaviour when CFW is on.
> > No, he said that if you disable CFW in the control unit but enable it
in
> > DFSORT, the control unit ignores the CFW request and treats it as a
> > regular write.
> >
> > --
> > Bruce A. Black
> > Senior Software Developer for FDR
> > Innovation Data Processing 973-890-7300
> > personal: [EMAIL PROTECTED]
> > sales info: [EMAIL PROTECTED]
> > tech support: [EMAIL PROTECTED]
> > web: www.innovationdp.fdr.com
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> > Search the archives at http://bama.ua.edu/archives/ibm-main.html
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
> Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: CFW and DFSORT

[David Betten]

You should not have a problem with DFSORT if you disable cache fast write
in your contoller.  Even if the DFSORT default is CFW, the controller will
simply ignore the CFW flag and treat it as a normal write.  Generally, I
have not seen a significant perfomrance impact to DFSORT when CFW is turned
off.  This is mainly due to the larger NVS sizes and high speed interfaces
of today's storage technology.



Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Mainframe Stress Testing

[David Betten]

Loadrunner actually does have a 3270 interface now so it can drive terminal
traffic to a mainframe CICS application.   I believe it can simulate
multiple users using a tn3270 emulator to access the mainframe application.

No matter what tool you use, the real work is in developing an accurate
workload.  You have to make sure you have a realistic transaction mix and
you have to get valid data to feed into the scripts.  This usually involves
data base queries to generate lists of valid acct numbers, etc.   Then once
you get it all working, you have to keep the data and the scripts current.




Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
email:  [EMAIL PROTECTED]
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html