Chris:

When I worked at a major publishing company (40 years ago+) we acquired SYNCSORT and at least weekly we sorted our master file (not sure I can tell you why either) and during the sort we used to sort 25 + million customer records (lots of other big sorts as well) we saved quite a bit of time by purchasing SYNCSORT and it actually supported E61 (which the other sort did not (CA?) ) we were pretty much a happy SYNCSORT user until IBM seriously started to optimize the sort. It seemed we would benchmark the two sorts every few years until it was a dead heat and then DFSORT started saving us real elapsed (and CPU) time. It was not all happy time with DFSORT as the first time we tried to use the "improved" DFSORT there was a serialization storage issue on an MP. Two years later we tried again and the issue apparently was fixed and we decided to go back to DFSORT. One issue with SYNCSORT was that the fixes they sent out were in the form of ZAP's and we were constantly running out of IDR space.
Also the zap were prone to keying issues (mine).
I liked IBM's SMP philosophy and had wished SYNCSORT had adopted it and I have learned since that they (Syncsort) has moved on to the 20th century. I will also admit SYNCSORT was as good as any at the time and getting help with an issue was never an issue (wish other vendors were half as good).
Now days DFSORT is pretty solid as well and support is pretty good.
As we have seen on here Syncsort has a good presence on IBM and as well DFSORT.

Ed

On Oct 9, 2012, at 10:16 AM, Blaicher, Christopher Y. wrote:

Yes, you could say that, but there are trade-offs.

When dealing with the EQUALS option, a field is internally added to each record by the sort. That is more data that has to be carried around internally and on the SORTWK files.

Consider that to accommodate over 4 billion rows, that increased the DASD requirements by at least 8G. Please note - I made an error in my previous post. DB2 Sort and SyncSort can accommodate 2**48 records, not 2**40 records.

When the amount of data approaches 256 trillion records, I am sure we will accommodate it. Will we jump to a full 2**64? Probably, but that remains to be seen.

When I started with Syncsort, a 100 MegaByte sort was BIG and a 4 billion record sort was unthinkable. Now a 100M sort can be an in- core sort. What will the future bring? I certainly don't know, but at the rate data is growing, in the not too distant future we will look at a 4 billion record sort and consider it normal.

Chris Blaicher
Senior Software Engineer, Software Services
Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8260  |  M: 512-627-3803
E: cblaic...@syncsort.com


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM- m...@listserv.ua.edu] On Behalf Of Rob Schramm
Sent: Tuesday, October 09, 2012 9:50 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Reorganizing Partition Table Space With REBALANCE Option Problem.

Considering how data is growing, don't these limitations seem a bit arbitrary? 2**32 or 2**40?

Rob Schramm
On Oct 9, 2012 10:01 AM, "Blaicher, Christopher Y." <cblaic...@syncsort.com>
wrote:

DB2 Sort and SyncSort do not have that limitation.  Their limitation
is
2**40 rather than 2**32.

Chris Blaicher
Senior Software Engineer, Software Services Syncsort Incorporated
50 Tice Boulevard, Woodcliff Lake, NJ 07677
P: 201-930-8260  |  M: 512-627-3803
E: cblaic...@syncsort.com


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM- m...@listserv.ua.edu] On
Behalf Of Mehrshad Manshadi
Sent: Tuesday, October 09, 2012 1:39 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Reorganizing Partition Table Space With REBALANCE Option
Problem.

Hi David,

Thanks for your useful information.

So The APAR PM43006 will solve our problem? Am I right?

Best regards
Manshadi




________________________________
 From: David Betten <bet...@us.ibm.com>
To: IBM-MAIN@LISTSERV.UA.EDU
Sent: Monday, October 8, 2012 6:01 PM
Subject: Re: Reorganizing Partition Table Space With REBALANCE Option
Problem.

Unfortunately, you are exceeding a DFSORT limitation on the number of
records that can be sorted.  DFSORT has the following limitations

1. Variable-length records - 4294967295 records 2. EQUALS in effect -
4294967295 records 3. VLSHRT in effect - 2147483647 records 4. Blockset
technique not selected - 2147483647 records


Number 4 will not be a problem since Blockset is always used for sorts invoked by DB2 Utilities. DB2 APAR PM43006 causes the utilities to pass NOVLSHRT and NOEQUALS which would resolve numbers 2 and 3 but since this is a vaiable length sort, you're still going to have that first limitation of
4,294,967,295 records.  Normally you'd want to break up the reorg by
partitions but since you're doing a rebalance, I guess that's why it's trying to sort the entire file at once. I suggest you open PMR and see if
DB2 Utilities support can provide a solution.

Also two additional notes about this.

1. It looks like your DFSORTinstallation default for DSA is 64 and that's causing the utility to limit the mainsize it passes. You probably want something much larger for a sort this size. I'd suggest changing your
installation default to something much larger like DSA=256.
2. It appears you are running DFSORT V1R5. That has been out of service
for quite some time.



Have a nice day,
Dave Betten
DFSMS Performance Engineer
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu> wrote on
10/08/2012 10:10:10 AM:

From: Mehrshad Manshadi <m_mansh...@yahoo.com>
To: IBM-MAIN@listserv.ua.edu,
Date: 10/08/2012 10:11 AM
Subject: Re: Reorganizing Partition Table Space With REBALANCE Option
Problem.
Sent by: IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu>

Hi Dave,



The attached is sort message of the output job.


1ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED

ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS,
EXAMPLES A
 ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 11:
18 ON MON OC
0              OPTION MOSIZE=0
 ICE140I 0 END OF PARAMETERS FROM DFSPARM  - SYSIN OR SORTCNTL/
PARAMETER LIST CO
           SORT FIELDS=(00005.0,00028.
0,A),FORMAT=BI,FILSZ=E000004979426640,DYNA
                          OC=(SYSDA,04)
           RECORD TYPE=V,LENGTH=(00070,00070,00070)
           OPTION
MSGPRT=ALL,MSGDDN=UTPRINT,AVGRLEN=00070,MAINSIZE=065536K
 ICE201I 0 RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE751I 0 C5-BASE   C6-BASE   C7-BASE   C8-Q83041 E4-BASE   C9-BASE
  E5-Q90312
 ICE193I 0 ICEAM2 ENVIRONMENT IN EFFECT - ICEAM2 INSTALLATION MODULE
SELECTED
 ICE089I 0 ROINHORB.REORG   .DSNUPROC, INPUT LRECL = 70, TYPE = V
 ICE092I 0 MAIN STORAGE = (67108864,67108864,67108864)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (66060288,66060288)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0
,SPANINC=RC16,VLSCMP=N,SZERO=Y
 ICE128I 0 OPTIONS:
SIZE=67108864,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,E
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO
,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHA
 ICE130I 0 OPTIONS: RESALL=0,RESINV=0,SVC=109
,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,
 ICE131I 0 OPTIONS:
TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW
 ICE132I 0 OPTIONS:
VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXIT
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX
,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAM
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE750I 0 DC 0 TC 0 CS DSVXX KSZ 32 VSZ 32
 ICE752I 0 FSZ=4979426640 RE  IGN=0 C  AVG=70 70  WSP=452719355 E
DYN=0
0
 ICE121A 7 FILE SIZE IS TOO LARGE
 ICE751I 1 D8-Q83041 D4-Q84357 E8-BASE
 ICE052I 0 END OF DFSORT


Best regards
Manshadi



________________________________
 From: David Betten <bet...@us.ibm.com>
To: IBM-MAIN@LISTSERV.UA.EDU
Sent: Monday, October 8, 2012 4:05 PM
Subject: Re: Reorganizing Partition Table Space With REBALANCE Option
Problem.

You need to look at the sort messages to determine why the sort failed.
If
you want, send the entire joblog to the DFSORT hot line
(dfs...@us.ibm.com)
and we can look it over for you.


Have a nice day,
Dave Betten
DFSMS Performance Engineer
IBM Corporation
email:  bet...@us.ibm.com
1-301-240-3809
DFSORT/MVSontheweb at http://www.ibm.com/storage/dfsort/

IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu> wrote on
10/08/2012 08:03:35 AM:

From: Mehrshad Manshadi <m_mansh...@yahoo.com>
To: IBM-MAIN@listserv.ua.edu,
Date: 10/08/2012 08:06 AM
Subject: Reorganizing Partition Table Space With REBALANCE Option
Problem.
Sent by: IBM Mainframe Discussion List <IBM-MAIN@listserv.ua.edu>

Hi,

We have a table space with 250 partition and 4,600,000,000 records
which it's have to re partition.

Because of lack space on the DASD we are trying to use Tape on some
DD cards as follows :

//SYSREC    DD DSN=TEMPDS.TAPE1,DISP=(MOD,CATLG),
//          UNIT=CART,RETPD=36,BLKSIZE=0,LABEL=(,SL)
//SYSUT1    DD DSN=TEMPDS.TAPE2,DISP=(NEW,CATLG),
//          UNIT=CART,RETPD=36,BLKSIZE=0,LABEL=(,SL)
//SORTOUT   DD
 DSN=TEMPDS.TAPE3,DISP=(NEW,CATLG),
//          UNIT=CART,RETPD=36,BLKSIZE=0,LABEL=(,SL)
//SYSCOPY   DD DSN=TEMPDS.TAPE4,DISP=(NEW,CATLG),
//          UNIT=CART,RETPD=36,BLKSIZE=0,LABEL=(,SL)

We are using 123 work data sets as follows :

//SORTWK01  DD DISP=(NEW,DELETE),UNIT=SYSDA, //
SPACE=(CYL,(4300,1000),RLSE), //
VOL=(,,,180),STORCLAS=SCLARGE
//SORTWK02  DD DISP=(NEW,DELETE),UNIT=SYSDA, //
SPACE=(CYL,(4300,1000),RLSE), //
VOL=(,,,180),STORCLAS=SCLARGE
//SORTWK03  DD DISP=(NEW,DELETE),UNIT=SYSDA, //
SPACE=(CYL,(4300,1000),RLSE), //
  VOL=(,,,180),STORCLAS=SCLARGE

.
.
.
.
.
.
//SORTWK4F  DD DISP=(NEW,DELETE),UNIT=SYSDA, //
SPACE=(CYL,(4300,1000),RLSE), //
VOL=(,,,180),STORCLAS=SCLARGE

And the reorg options as follows :

//DFSPARM  DD  *
    OPTION MOSIZE=0
//DSNUPROC.SYSIN DD *
REORG TABLESPACE QQBANCS.TSINHO REBALANCE
    LOG NO SORTDATA SORTDEVT SYSDA SORTNUM 4
    COPYDDN(SYSCOPY)
/*

The problem is the unload phase cont the number of records which we
expected and after this we assume that it should be  switch to next
step , but the job failed with following error:

DSNUGUTC - OUTPUT START FOR UTILITY, UTILID = ROINHORB DSNUGTIS -
 PROCESSING SYSIN AS EBCDIC
DSNUGUTC -  REORG TABLESPACE QQBANCS.TSINHO REBALANCE LOG NO
SORTDATA

DSNUGSRT - ERROR FROM SORT COMPONENT RC=16, UTILITY STOPPED DSNUGBAC
- UTILITY BATCH MEMORY EXECUTION ABENDED, REASON=X'00E40005'

Please help if you have any idea about it.

Best regards
Manshadi

------------------------------------------------------------------- -
-- For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO
IBM-MAIN

-------------------------------------------------------------------- --
For IBM-MAIN subscribe / signoff / archive access instructions, send
email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

-------------------------------------------------------------------- --
For IBM-MAIN subscribe / signoff / archive access instructions, send
email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--------------------------------------------------------------------- - For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--------------------------------------------------------------------- - For IBM-MAIN subscribe / signoff / archive access instructions, send email
to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



ATTENTION: -----

The information contained in this message (including any files transmitted
with this message) may contain proprietary, trade secret or other
 confidential and/or legally privileged information. Any pricing
information contained in this message or in any files transmitted with this message is always confidential and cannot be shared with any third parties without prior written approval from Syncsort. This message is intended to be read only by the individual or entity to whom it is addressed or by
their designee. If the reader of this message is not the intended
recipient, you are on notice that any use, disclosure, copying or
distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or Syncsort and destroy all copies of this message in your possession,
custody or control.

--------------------------------------------------------------------- -
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM- MAIN


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Reply via email to