Db2 Utilities is the only exception to the program invoked restriction.
And reading the APAR, you'll notice even that is limited to "variable
length data sort of the REORG TABLESPACE utility". The reason for these
restrictions is that data has to be fed into the sort at a high speed to
realize
It has to do with the order of the operations. OMIT is done on input,
prior to the summing. OUTFIL happens on output, after the summing.
Consider four records
key Include field
KEY1 M
KEY1 P
KEY1 M
KEY1 Q
With SUM and OMIT
OMIT will remove the M records 1 and 3 on input leaving the KEY1
If you want to let DFSORT determine the amount of work space to allocate,
you need to remove the JCL SORTWORK DDs. Once DFSORT sees you've coded
your own SORTWORKs, it uses whatever space you've hard coded. I'd also add
OPTION DYNALLOC=(SYSDA,16) to your sort control statements. Instead of
OW0007OB is only 2 bytes followed by 2 unused bytes.
The reason is that OQ0007AC is defines as a full word and thus has to be
full word aligned at address 000804. If you had defined OW0007AC as just
XL4 it would've started at 000802. Similarly, if you had defined it as D,
then it would have to
DB2 Utilities are a good example of this. They often execute multiple
concurrent sorts. You use SORTDD= in the parameter lists to pass a
separate prefix for each sort. For example you might pass SORTDD=SRT1 for
the first one, SORDD=SRT2 for the second, etc. That way each will allocate
SRT1WK*,
Just for clarification, I've been working with Shivang on this and he's
since opened a PMR on the problem. When his job uses VIO even for a very
small file, it automatically pages out for the writes and pages in for the
reads. With RMF we clearly see there is plenty of available memory.
We've
If EQUALS is in effect, then DFSORT will always preserve the original order
of records with the same key value. If EQUALS is not in effect, then it is
possible for the sequence of records with the same key value to vary. One
reason this can vary is due to the amount of memory available on the
I haven't been in DFSORT for a while but I'm not aware that DFSORT made any
changes to read multi-volume input in parallel. What in the output makes
you think it is? The only way I know to get parallelism on the input is to
make the data set multi-stripe.
Have a nice day,
Dave Betten
z/OS
Just a few thoughts on ways you might be able to reduce the sort work
requirement.
Since the input is on tape, do we know if DFSORT is getting accurate file
size information? Do you see DFSORT messages in the output like UNKNOWN
FILE SIZE or INTERMEDIATE MERGE...?
The reason I ask is that if
I would suggest looking at an RMF workload activity report to see what the
workload is doing now rather than arbitrarily changing period durations and
goals. The workload activity report can tell you the average response time
and transactions rates, along with the service units and velocity as
This might help. On the DFSORT web site there's a file that includes
examples for using ICETOOL to analyze DFHSM, DFSMSrmm and DCOLLECT.
http://www-01.ibm.com/support/docview.wss?rs=114=isg3T797#ex
Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM
Again I'm somewhat guessing but I'd assume any promotion (BLK, ENQ, CRM,
LCK, SUP). I only mentioned local lock because I saw the additional field
of SMF99_HdLockPromotion_Time_at_PDP and SMF99_HdLock_Time_at_PDP_Using.
Hopefully someone with more expertise than me can confirm. Or if you're
I'm not 100% sure but that could be "promoted dispatching priority" where
SRM temporarily promotes an address space to a DP of X'FF' while it holds a
local lock.
Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email: bet...@us.ibm.com
I see a couple things the DFSORT output
The reason for the small mainsize and DSA=0 is because the program is
passing MAINSIZE=00128000
ICE146I 0 END OF STATEMENTS FROM SORTCNTL - PARAMETER LIST STATEMENTS
FOLLOW
SORT FIELDS=(0001,0013,CH,A)
RECORD TYPE=F,LENGTH=(49,,)
> Technology Solution Services
>
> 123 East Main Street
> Louisville, KY 40202
> Humana.com
> (502) 476-2538 or 407-7266
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU
> ] On Behalf Of David Betten
> Sent: Tu
Is it possible your installation default is EQUALS=YES? This causes extra
bytes to be added to the sort field internally. Try adding OPTION NOEQUALS
after your SORT statement.
Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email:
I think USING is a parameter of the SELECT but ICETOOL is seeing it as a
separate statement. Change the end of your SELECT statement to include a
continuation character.
//TOOLINDD *
SELECT FROM(SORTIN) TO(SORTF1) ON(16,50,CH) DISCARD(SORTF2) FIRST -
USING(CTL1)
Have a nice day,
Dave
As I understand it, NUP is the change since the data set was created where
as DUP is the change since open. I've always used DEP, DRE, DIN, DDE and
DUP to determine activity by a specific job step.
As an fyi, the type 42-6 records have a fairly new field which is the read
I/O count, so you can
The dynamic allocation space calculations are going to be based on the
DYNALLOC number. As a simple example, if DFSORT calculates that it needs
6,000 cylinders of work space, and DYNALLOC=4 with DYNAPCT=50, it will need
4 volumes with at least 1500 cylinders of free space. But with DYNALLOC=6
om/support/docview.wss?uid=isg3T165
Have a nice day,
Dave Betten
z/OS Performance Specialist
Cloud and Systems Performance
IBM Corporation
email: bet...@us.ibm.com
1-720-396-3882
IBM Mainframe Discussion List <IBM-MAIN@LISTSERV.UA.EDU> wrote on
06/20/2016 10:30:08 AM:
> From: David Bett
I may be mistaken but I don't think Dynamic Volume Count is the real issue
with using up TIOT entries. What I'd be more concerned about is Volume
Count as if there's a high number like 59, a TIOT entry will be created for
each candidate volume.Dynamic Volume Count is only a factor if Space
When I was in DFSORT, I always recommended not using VIO for sort work data
sets. We felt it was better to let DFSORT manage the use of storage for
intermediate work space via Hiperspace, Memory Object or Dataspace. One
advantage was that DFSORT has controls over the total memory usage by
You might consider testing with PDSE. In z/OS 2.1 there were a number of
performance improvements made to PDSE processing including improved
directory processing. If you do a google search, you'll find a number of
Share presentations on the PDSE improvements.
Have a nice day,
Dave Betten
z/OS
First of all, full disclaimer that I was in DFSORT development for about 8
years so I might be biased. But I just want to share a few thoughts.
First the idea of loading all the data into a large hashmap to do the sort
tends to eliminate one very important thing and that's overlap.
Essentially,
I agree with Martin about Type 42. At the end of a recording interval,
only one record is cut for each job and then there are multiple data set
headers with each one containing an offset to the next. So that would be
very difficult to process without some sore of E15 exit to break those up
and
Without getting too far into the DFSORT internals, I'll just clarify that
DFSORT generally prefers to use memory objects instead of Hiperspaces.
However, rather than remove the Hiperspace support we kept it there. Part
of the reason is that often times the memlimit may prevent the use of
memory
I'm not in the DFSORT team anymore but thought I'd chime in here. File
size and dynamic allocation is a topic I could probably spend hours trying
to explain since there are so many nuances. But to try and answer your
specific questions.
1. When DFSORT can determine the file size, it calculates
I there a blank line between SYSMS2 and LOGDD2 statements?
Have a nice day,
Dave Betten
z/OS Performance Specialist - HiPODS
Open Technology and Cloud Performance
IBM Corporation
email: bet...@us.ibm.com
1-301-240-3809
IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU wrote on
05/27/2015
EXPMAX limit applies to total storage used by concurrent sorts. Usually
when excessive paging happens it's in environments like DB2 where there's
a high number of old frames in central storage that have not been
recently referenced. DFSORT includes those frames in its calculations of
storage
There are many variables that we don't know from the information you
provided. For a SORT, we have to allocate many other areas of virtual
storage for sortwork buffers, etc. Also, I don't know which access method
DFSORT is using for your SORTIN, what release of DFSORT, etc. Your best
bet is
at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List IBM-MAIN@listserv.ua.edu wrote on
07/18/2014 06:26:32 PM:
From: David Betten/Gaithersburg/IBM@IBMUS
To: IBM-MAIN@listserv.ua.edu
Date: 07/18/2014 06:27 PM
Subject: Re: DFSORT Performance
Sent by: IBM Mainframe Discussion List IBM
DFSORT does look at available resources before it grabs whatever it can.
There are DFSORT installation defaults to control how much of the
available storage DFSORT will use but I agree the shipped defaults can
cause issues in some environments. We put a lot of guidance on these
installation
[mailto:IBM-MAIN@LISTSERV.UA.EDU]
On Behalf Of David Betten
Sent: Friday, April 18, 2014 10:31 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SORT ando MEMLIMIT best practice
DFSORT does look at available resources before it grabs whatever it
can.
There are DFSORT installation defaults
I've been working on the ETR with our L2 support and waiting for some doc
to confirm my assumptions. But I figure I better update this thread since
so many seem to be speculating. I believe what happened is this.
One of the things DFSORT looks at is the values returned from SYSEVENT
STGTEST.
Anne,
I've been discussing this with the DFSORT L2 rep that's working on
your problem. I don't believe CFW is related to this. I think I have an
idea of what's going on, but I'd rather work the problem through our
support process and then once we are sure of the cause, I can post an
Discussion List IBM-MAIN@listserv.ua.edu
Hi David,
Thanks for your useful information.
So The APAR PM43006 will solve our problem? Am I right?
Best regards
Manshadi
From: David Betten bet...@us.ibm.com
To: IBM-MAIN@LISTSERV.UA.EDU
Sent: Monday
You need to look at the sort messages to determine why the sort failed. If
you want, send the entire joblog to the DFSORT hot line (dfs...@us.ibm.com)
and we can look it over for you.
Have a nice day,
Dave Betten
DFSMS Performance Engineer
IBM Corporation
email: bet...@us.ibm.com
7 FILE SIZE IS TOO LARGE
ICE751I 1 D8-Q83041 D4-Q84357 E8-BASE
ICE052I 0 END OF DFSORT
Best regards
Manshadi
From: David Betten bet...@us.ibm.com
To: IBM-MAIN@LISTSERV.UA.EDU
Sent: Monday, October 8, 2012 4:05 PM
Subject: Re: Reorganizing Partition
Based on the record statement and the FILSZ specified, DFSORT estimated it
would need 469577MB of disk work space. Your installation default for
DYNALOC is probably the shipped default of SYSDA,4. This is going to cause
DFSORT to allocate 4 work data sets. The work space allocation will be
Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
Behalf Of David Betten
Sent: Mittwoch, 3. Oktober 2012 09:59
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Help with DF-Sort not being able to allocat enough work
data set space
Based on the record statement and the FILSZ specified, DFSORT
-09-12 16:08, David Betten pisze:
Just an fyi, that in DFDORT V1R13 we added a unit count of 1 to the
dynamic
allocation parameters to override any data class specification of a
unit
count.
Just to be sure: you mean DFSORT for z/OS 1.13?
I'm asking because, it can be confused with DFSORT
I do not believe VLSHRT requires NOBLKSET. I would try removing NOBLKSET
from the sort and try again.
Have a nice day,
Dave Betten
DFSMS Performance Engineer
IBM Corporation
email: bet...@us.ibm.com
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/
IBM Mainframe Discussion List
I personally prefer using a volume count of 1 but then setting Space
Constraint Relief to YES with a dynamic volume count. That way my small
files are allocated with 1 volume and additional volumes are only allocated
if needed.
Have a nice day,
Dave Betten
DFSORT Development, Performance Lead
This is hard to resolve without knowing where the error is occuring, It
could be something like ICE083A at the very beginning or it could be
running a while and getting ICE046A. If you can get the sysout or at least
some of the messages, that would help. For now I have a few
recommendaitons.
44 matches
Mail list logo
