Re: SDSF - so slow!
OK Thanks Andy, I'll give it gao when I have some time to have a play. Cheers Gerry Anstey Andy Wood To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List Re: SDSF - so slow! 04/02/2009 23:13 Please respond to IBM Mainframe Discussion List This sounds suspiciously like something I saw with SDSF a long time ago. I like to think that things would have improved by now, but who knows? This is how you could tell: If you don't have the original job still on spool run a job to create a test SYSOUT dataset. Use something that will write records that don't contain lots of blanks, and the more records the better - several million would be good. Create a job to run SDSF in batch to select your test SYSOUT and capture it to a DASD dataset (or even to DD DUMMY), using the SDSF PRINT command, but set it up to not capture the whole test dataset - use something like "PRINT 1 50". Run your SDSF batch job a number of times, varying the number of records printed, by changing the second number of the PRINT range. Try it for say 50, 100, 150 etc and see how the resource usage, particularly CPU time increases as it processes more records. If you find that CPU time increases disproportionally with increasing number of records processed, you probably have the same problem I found, where it became unusable for very large datasets. If it is still like that you could try opening a PMR. If they tell you that you are the only one to ever have this problem, suggest that they search the PMR archive for about 1998, looking for the words ISFDSRC and PARROT. Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
This sounds suspiciously like something I saw with SDSF a long time ago. I like to think that things would have improved by now, but who knows? This is how you could tell: If you don't have the original job still on spool run a job to create a test SYSOUT dataset. Use something that will write records that don't contain lots of blanks, and the more records the better - several million would be good. Create a job to run SDSF in batch to select your test SYSOUT and capture it to a DASD dataset (or even to DD DUMMY), using the SDSF PRINT command, but set it up to not capture the whole test dataset - use something like "PRINT 1 50". Run your SDSF batch job a number of times, varying the number of records printed, by changing the second number of the PRINT range. Try it for say 50, 100, 150 etc and see how the resource usage, particularly CPU time increases as it processes more records. If you find that CPU time increases disproportionally with increasing number of records processed, you probably have the same problem I found, where it became unusable for very large datasets. If it is still like that you could try opening a PMR. If they tell you that you are the only one to ever have this problem, suggest that they search the PMR archive for about 1998, looking for the words ISFDSRC and PARROT. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
So, perhaps opening a PMR against SDSF for performance issues in reading the spool files would be in line. Usually this works with IBM but SDSF folks seem to take pleasure in not incorporating functionality nor addressing issues/problems but then again I was use to (E)JES before this shop. Jack Kelly 202-502-2390 (Office) -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Brian, here is the the file after, it's blocked efficiently, indeed other things process it in minutes: Data Set Information Command ===> Data Set Name . . . . : SZ.SDSF.WORK General Data Current Allocation Management class . . : DEVSTD Allocated cylinders : 4,015 Storage class . . . : STANDARDAllocated extents . : 201 Volume serial . . . : HAT10D + Device type . . . . : 3390 Data class . . . . . : MULTDVECurrent Utilization Organization . . . : PS Used cylinders . . : 4,012 Record format . . . : FBA Used extents . . . : 201 Record length . . . : 133 Block size . . . . : 27930 1st extent cylinders: 15 Secondary cylinders : 20 Data set name type : SMS Compressible : NO Creation date . . . : 2009/02/03 Referenced date . . : 2009/02/03 Expiration date . . : ***None*** To display multiple volumes press Enter or enter Cancel to Exit. Brian Westerman To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List Re: SDSF - so slow! 04/02/2009 06:03 Please respond to IBM Mainframe Discussion List I can't believe how stupid my last comment (and test) was, I looked at the wrong line of his output and completely missed that the next DS in line was 25 MILLION lines. I don't have any 25M line jobs to test, but I went to our test runs of SyzSpool and looked at the stats for various size output. I don't see how any of them would multiply out to the numbers that he is seeing. The 120,000 EXCPs is possible putting about 400 recs/block in the datasets, but that seems to be strange (or maybe pretty big records). Maybe your writing them out to a dataset that is blocked badly, or the JES dataset is really screwed up. If you were to select your output in SDSF, then MAX DOWN to the bottom, how long does that take? If it isn't 1/2 of the numbers that you have provided initially, then the problem isn't with the input dataset (or SDSF) itself, but rather with the output DS that you are writing it into. Although if it's the input DS that is the problem then this test will not really show us very good results, only if the output dataset is the issue. It would be interesting to know the record size of the input and the DCB of the output. Sorry for the stupidity of the last post. Brian Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase
Re: SDSF - so slow!
Frank,
write them to standard DASD, and if necessary IEBGENER the DS to SYSUT2=*
as well then you can have them on the spool also. Several have suggested
this and I had already recomeneded that too it's just that I was mainly
curious at what SDSF was doing to be so dam slow. In my case I wasn't even
doing a "FIND" I was just just doing a PRINT ODSN as described in my
original post.
GA
Frank Swarbrick
To
Sent by: IBM IBM-MAIN@bama.ua.edu
Mainframe cc
Discussion List
Re: SDSF - so slow!
03/02/2009 18:58
Please respond to
IBM Mainframe
Discussion List
On Tue, 3 Feb 2009 11:25:05 +, Gerry Anstey
wrote:
>OK bad design, we have lot of cack here, probably due to hiring cheap
>programmers, any way, I digress.
>
>Here s the SDSF summary:
>
> PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS
> NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT
> JESMSGLG JES2 2 PSTGCS T LOCAL 52
> JESJCL JES2 3 PSTGCS T LOCAL 44
> JESYSMSG JES2 4 PSTGCS T LOCAL 94
> DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3
> CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559
> CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M
>
>Now we had a need to extract some of the records in CMPRT01, I wrote job
to
>run SDSF in batch and to use the PRINT ODSN command to extract the data
to
>a data set.
>
>Then I read the dataset with Filemaster and extracted the desired records
>into a smaller file.
>
>My questions are:
>
>1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins
CPU) to
>read and write out the data and Filemaster takes about 3 minutes to read
>25million recs and write about 1.5million?
>
>2. Is there any way to make SDSF extract faster?
Hope you don't mind me stealing your thread, but it made me think of
something we need to deal with.
We are on the front end of a project to move from VSE to z/OS. We
currently
archive our reports in IBM Content OnDemand (Windows version, not z/OS
version).
Basically we write all of our reports to the VSE/POWER list queue and then
we
run a job that FTPs them out of the list queue to the OnDemand server,
which
then loads them in to OnDemand.
Can you FTP from the JES spool? And even if you can, is it a good idea?
Or
should we just go ahead and write them to sequential disk files instead of
SYSOUT?
Frank
Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation
of any transaction. In the event you are receiving the offering
materials attached below related to your interest in hedge funds or
private equity, this communication may be intended as an offer or
solicitation for the purchase or sale of such fund(s). All market
prices, data and other information are not warranted as to
completeness or accuracy and are subject to change without notice.
Any comments or statements made herein do not necessarily reflect
those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the respons
Re: SDSF - so slow!
I can't believe how stupid my last comment (and test) was, I looked at the wrong line of his output and completely missed that the next DS in line was 25 MILLION lines. I don't have any 25M line jobs to test, but I went to our test runs of SyzSpool and looked at the stats for various size output. I don't see how any of them would multiply out to the numbers that he is seeing. The 120,000 EXCPs is possible putting about 400 recs/block in the datasets, but that seems to be strange (or maybe pretty big records). Maybe your writing them out to a dataset that is blocked badly, or the JES dataset is really screwed up. If you were to select your output in SDSF, then MAX DOWN to the bottom, how long does that take? If it isn't 1/2 of the numbers that you have provided initially, then the problem isn't with the input dataset (or SDSF) itself, but rather with the output DS that you are writing it into. Although if it's the input DS that is the problem then this test will not really show us very good results, only if the output dataset is the issue. It would be interesting to know the record size of the input and the DCB of the output. Sorry for the stupidity of the last post. Brian -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Hi, I don't think it's SAPI, our spool dataset and report offload product (SyzSpool) uses SAPI and the speed is far greater than what he seems to be experiencing. SAPI isn't used to actually get each record, only to get access to the JES dataset name, the dynamic allocation is performed and transfer is simple QSAM. SDSF uses the same method. I tried the test here at z/OS 1.9 on a JES DS with 280,000+ records in it (10 times his number) and I was able to have SDSF put it into a sequential dataset in less than a minute. I don't have the exact time because I expected it to be slower and went to get a can of diet Coke and it was done before I got back. Not very scientific, but I was thirsty. I think that there must be something else going on there that is worth investigating. Brian -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
On Tue, 3 Feb 2009 11:25:05 +, Gerry Anstey wrote: >OK bad design, we have lot of cack here, probably due to hiring cheap >programmers, any way, I digress. > >Here s the SDSF summary: > > PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS > NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT > JESMSGLG JES2 2 PSTGCS T LOCAL 52 > JESJCL JES2 3 PSTGCS T LOCAL 44 > JESYSMSG JES2 4 PSTGCS T LOCAL 94 > DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 > CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 > CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M > >Now we had a need to extract some of the records in CMPRT01, I wrote job to >run SDSF in batch and to use the PRINT ODSN command to extract the data to >a data set. > >Then I read the dataset with Filemaster and extracted the desired records >into a smaller file. > >My questions are: > >1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to >read and write out the data and Filemaster takes about 3 minutes to read >25million recs and write about 1.5million? > >2. Is there any way to make SDSF extract faster? Hope you don't mind me stealing your thread, but it made me think of something we need to deal with. We are on the front end of a project to move from VSE to z/OS. We currently archive our reports in IBM Content OnDemand (Windows version, not z/OS version). Basically we write all of our reports to the VSE/POWER list queue and then we run a job that FTPs them out of the list queue to the OnDemand server, which then loads them in to OnDemand. Can you FTP from the JES spool? And even if you can, is it a good idea? Or should we just go ahead and write them to sequential disk files instead of SYSOUT? Frank -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Paul Gilmartin wrote: Wouldn't FTP use the same SSI calls as SDSF, with similar performance? Possibly not. FTP is designed only to read beginning to end. SDSF is designed to allow scrolling up and down, locating specific records, backward find processing, etc. Keep in mind that SDSF spool access has *always* been comparatively slow. I'm pretty sure they used to use the IAZSPLIO service (SSI 71) to read spool blocks. At some point, I believe they switched everything over to using the spool data set browse (SDSB) facility to facilitate their JES neutrality solution (aka support for JES3). If so, that might have pushed them past the knee of the performance curve. The difference is probably not that noticeable for most smaller data sets. (E)JES uses a STARTIO I/O driver and modern CCW commands to read whole spool tracks in a single operation, even while our code executes on a zIIP processor. Some of the other, non-IBM spool browsers might also use similar high-performance I/O techniques. -- Edward E Jaffe Phoenix Software International, Inc 5200 W Century Blvd, Suite 800 Los Angeles, CA 90045 310-338-0400 x318 edja...@phoenixsoftware.com http://www.phoenixsoftware.com/ -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
- OK bad design, we have lot of cack here, probably due to hiring cheap programmers, any way, I digress. Here s the SDSF summary: PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT JESMSGLG JES2 2 PSTGCS T LOCAL 52 JESJCL JES2 3 PSTGCS T LOCAL 44 JESYSMSG JES2 4 PSTGCS T LOCAL 94 DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M Now we had a need to extract some of the records in CMPRT01, I wrote job to run SDSF in batch and to use the PRINT ODSN command to extract the data to a data set. Then I read the dataset with Filemaster and extracted the desired records into a smaller file. My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? thanks - While I can't comment to your single case, I can tell you that,in general, processing data from the SPOOL can be pretty slow, because of all the "machinations" necessary to retrieve the data and get it where you need it. Let me suggest this: send the report output to a DASD file, then use IEBGENER to spool it. You ancillary processing can access the DASD file and you'll still have a SPOOL file you can print, or delete, as the situation dictates. -- Rick -- Remember that if you’re not the lead dog, the view never changes. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Obviously an Natural program, do you have Entire System Server aka Natural Process? A guy named Ray put some pretty good JES access in that product. You might have been able to do it in one pass and faster. Dave Gibney Information Technology Services Washington State Univsersity -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Gerry Anstey Sent: Tuesday, February 03, 2009 3:25 AM To: IBM-MAIN@bama.ua.edu Subject: SDSF - so slow! OK bad design, we have lot of cack here, probably due to hiring cheap programmers, any way, I digress. Here s the SDSF summary: PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT JESMSGLG JES2 2 PSTGCS T LOCAL 52 JESJCL JES2 3 PSTGCS T LOCAL 44 JESYSMSG JES2 4 PSTGCS T LOCAL 94 DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M Now we had a need to extract some of the records in CMPRT01, I wrote job to run SDSF in batch and to use the PRINT ODSN command to extract the data to a data set. Then I read the dataset with Filemaster and extracted the desired records into a smaller file. My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? thanks Gerry Anstey Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
-Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Mary Anne Matyaz Sent: Tuesday, February 03, 2009 8:58 AM To: IBM-MAIN@bama.ua.edu Subject: Re: SDSF - so slow! There is a recent apar for SDSF performance, but it's in the H, O displays for a large number of jobs. OA25498<https://www-304.ibm.com/ibmlink/psp/viewAparDoc.wss?context=apar &documentIds=OA25498&lc=en&cc=US> The item I reported is to a large degree summed up in PK51393, which was taken as SUG. The problem is that trying to get through a large spool file is slow. The FIND command is impossibly slow (even when doing a find previous!). IBM's suggested work around is to use limits so that you get control after x records have been processed. All this does is further increase the time to handle the spool file one is looking at (and in this case, this is not for the whole job but for a specific DD). Regards, Steve Thompson -- Opinions expressed by poster may not reflect those of poster's employer. -- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
On Tue, 3 Feb 2009 07:27:03 -0500, Rob Schramm wrote: >Not that this is much more than a slightly silly suggestion.. but you >could use FTP to get the output. > Wouldn't FTP use the same SSI calls as SDSF, with similar performance? OTOH, if FTP performs significantly better than SDSF, it provides support for the Requirement to improve SDSF performance, and guidance for an implementation technique. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
There is a recent apar for SDSF performance, but it's in the H, O displays for a large number of jobs. OA25498<https://www-304.ibm.com/ibmlink/psp/viewAparDoc.wss?context=apar&documentIds=OA25498&lc=en&cc=US> SDSF H display at z/OS 1.9 is slow. Starting at 1.9, SDSF uses the Extended Status SSI call to gather the display data. Dump and trace analysis shows that for the extended status SSI call, STATOUTT request in IAZSSST with jobname filter shows extensive processing retrieving JQAs for each job before applying filters, particularly jobname filter. The overhead of DOGJQE read is driven for every job in the system before filters such as jobname are applied which may not need the full JQA. This results in an inefficient extended status request. MA On Tue, Feb 3, 2009 at 9:41 AM, Thompson, Steve wrote: > -Original Message- > From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On > Behalf Of Gerry Anstey > Sent: Tuesday, February 03, 2009 5:25 AM > To: IBM-MAIN@bama.ua.edu > Subject: SDSF - so slow! > > > > My questions are: > > 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) > to > read and write out the data and Filemaster takes about 3 minutes to read > 25million recs and write about 1.5million? > > 2. Is there any way to make SDSF extract faster? > > > > I think you are talking about a problem that I brought to IBM's > attention about a year ago now. > > I'm going to paraphrase what was said: The more people complain, the > faster it will get fixed because management will take more notice. > > So, perhaps opening a PMR against SDSF for performance issues in reading > the spool files would be in line. > > Regards, > Steve Thompson > > -- Opinions expressed by poster may not be the opinions of poster's > employer -- > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO > Search the archives at http://bama.ua.edu/archives/ibm-main.html > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
-Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Gerry Anstey Sent: Tuesday, February 03, 2009 5:25 AM To: IBM-MAIN@bama.ua.edu Subject: SDSF - so slow! My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? I think you are talking about a problem that I brought to IBM's attention about a year ago now. I'm going to paraphrase what was said: The more people complain, the faster it will get fixed because management will take more notice. So, perhaps opening a PMR against SDSF for performance issues in reading the spool files would be in line. Regards, Steve Thompson -- Opinions expressed by poster may not be the opinions of poster's employer -- -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
ftp mybox
user xxx yyy
QUOTE SITE FILE=JES
DIR
DIR JOB12345
GET JOB12345.X
GET JOB12345.SYSPRINT
And if your systems programmer has set JESINTERFACELEVEL=2 in the FTP
parameters, you can:
QUOTE SITE JESOWNER=DJL*
QUOTE SITE JESJOBNAME=BUILD*
QUOTE SITE JESSTATUS=OUTPUT
And get listings for various and sundry owners, jobnames, MVS queues and the
like. As far as the get commands go, ".X" will get all outputs, ".yyy" will
get only the 'yyy' DD.
David Logan
Manager of Product Development, Pitney Bowes Business Insight
http://centrus.com
W: (720) 564-3056
C: (303) 818-8222
-Original Message-
From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf
Of Gerry Anstey
Sent: Tuesday, February 03, 2009 05:35
To: IBM-MAIN@bama.ua.edu
Subject: Re: SDSF - so slow!
Rob,
Not sure how I'd go about getting FTP to access the spool data.
Can you explain further, give and example etc
Thanks
GA
Rob Schramm
To
Sent by: IBM IBM-MAIN@bama.ua.edu
Mainframe cc
Discussion List
Re: SDSF - so slow!
03/02/2009 12:27
Please respond to
IBM Mainframe
Discussion List
Not that this is much more than a slightly silly suggestion.. but you
could use FTP to get the output.
Rob Schramm
Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation
of any transaction. In the event you are receiving the offering
materials attached below related to your interest in hedge funds or
private equity, this communication may be intended as an offer or
solicitation for the purchase or sale of such fund(s). All market
prices, data and other information are not warranted as to
completeness or accuracy and are subject to change without notice.
Any comments or statements made herein do not necessarily reflect
those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to UK legal entities.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Rob, Not sure how I'd go about getting FTP to access the spool data. Can you explain further, give and example etc Thanks GA Rob Schramm To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List Re: SDSF - so slow! 03/02/2009 12:27 Please respond to IBM Mainframe Discussion List Not that this is much more than a slightly silly suggestion.. but you could use FTP to get the output. Rob Schramm Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Not that this is much more than a slightly silly suggestion.. but you could use FTP to get the output. Rob Schramm -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
Re: SDSF - so slow!
Access to spool data is performed using an SSI call to JES (function 1 or 79 -
or sometimes called "SAPI") - you supply a parameter list to JES and it then
fetches the data for you and keeps track of where you are within the sysout
dataset so that you can ask for the next record. To achieve this correctly,
there is quite a lot of hoops that JES has to jump through to just return you
ONE record. I am not that familiar with SAPI but it is unlikely that its
performance can get anywhere near that of reading blocks of records from DASD
using QSAM.
Rob Scott
Rocket Software, Inc
275 Grove Street
Newton, MA 02466
617-614-2305
rsc...@rs.com
-Original Message-
From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of
Gerry Anstey
Sent: 03 February 2009 11:47
To: IBM-MAIN@bama.ua.edu
Subject: Re: SDSF - so slow!
Thanks Rob that was my advice too, It's not something we do a lot.
I was mainly curious as to what SDSF could possibly be doing to take so long,
surely the spool is just on DASD somewhere!
thanks
GA
Rob Scott
To
Sent by: IBM IBM-MAIN@bama.ua.edu
Mainframe cc
Discussion List
Re: SDSF - so slow!
03/02/2009 11:36
Please respond to
IBM Mainframe
Discussion List
Gerry,
SDSF has quite a bit of overhead to read records from the spool.
My advise would be to change the GCSPROCP job to write its output directly to a
DASD file and then pass it to the next step to be processed by Filemaster - if
you still need the output on JES spool then just add an extra IEBGENER step to
copy to SYSOUT.
Rob Scott
Rocket Software, Inc
275 Grove Street
Newton, MA 02466
617-614-2305
rsc...@rs.com
-Original Message-
From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of
Gerry Anstey
Sent: 03 February 2009 11:25
To: IBM-MAIN@bama.ua.edu
Subject: SDSF - so slow!
OK bad design, we have lot of cack here, probably due to hiring cheap
programmers, any way, I digress.
Here s the SDSF summary:
PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS
NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT
JESMSGLG JES2 2 PSTGCS T LOCAL 52
JESJCL JES2 3 PSTGCS T LOCAL 44
JESYSMSG JES2 4 PSTGCS T LOCAL 94
DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3
CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559
CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M
Now we had a need to extract some of the records in CMPRT01, I wrote job to run
SDSF in batch and to use the PRINT ODSN command to extract the data to a data
set.
Then I read the dataset with Filemaster and extracted the desired records into
a smaller file.
My questions are:
1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to
read and write out the data and Filemaster takes about 3 minutes to read
25million recs and write about 1.5million?
2. Is there any way to make SDSF extract faster?
thanks
Gerry Anstey
Generally, this communication is for informational purposes only and it is not
intended as an offer or solicitation for the purchase or sale of any financial
instrument or as an official confirmation of any transaction. In the event you
are receiving the offering materials attached below related to your interest in
hedge funds or private equity, this communication may be intended as an offer
or solicitation for the purchase or sale of such fund(s). All market prices,
data and other information are not warranted as to completeness or accuracy and
are subject to change without notice.
Any comments or statements made herein do not necessarily reflect those of
JPMorgan Chase & Co., its subsidiaries and affiliates.
This trans
Re: SDSF - so slow!
Thanks Rob that was my advice too, It's not something we do a lot. I was mainly curious as to what SDSF could possibly be doing to take so long, surely the spool is just on DASD somewhere! thanks GA Rob Scott To Sent by: IBM IBM-MAIN@bama.ua.edu Mainframe cc Discussion List Re: SDSF - so slow! 03/02/2009 11:36 Please respond to IBM Mainframe Discussion List Gerry, SDSF has quite a bit of overhead to read records from the spool. My advise would be to change the GCSPROCP job to write its output directly to a DASD file and then pass it to the next step to be processed by Filemaster - if you still need the output on JES spool then just add an extra IEBGENER step to copy to SYSOUT. Rob Scott Rocket Software, Inc 275 Grove Street Newton, MA 02466 617-614-2305 rsc...@rs.com -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Gerry Anstey Sent: 03 February 2009 11:25 To: IBM-MAIN@bama.ua.edu Subject: SDSF - so slow! OK bad design, we have lot of cack here, probably due to hiring cheap programmers, any way, I digress. Here s the SDSF summary: PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT JESMSGLG JES2 2 PSTGCS T LOCAL 52 JESJCL JES2 3 PSTGCS T LOCAL 44 JESYSMSG JES2 4 PSTGCS T LOCAL 94 DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M Now we had a need to extract some of the records in CMPRT01, I wrote job to run SDSF in batch and to use the PRINT ODSN command to extract the data to a data set. Then I read the dataset with Filemaster and extracted the desired records into a smaller file. My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? thanks Gerry Anstey Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether
Re: SDSF - so slow!
Gerry, SDSF has quite a bit of overhead to read records from the spool. My advise would be to change the GCSPROCP job to write its output directly to a DASD file and then pass it to the next step to be processed by Filemaster - if you still need the output on JES spool then just add an extra IEBGENER step to copy to SYSOUT. Rob Scott Rocket Software, Inc 275 Grove Street Newton, MA 02466 617-614-2305 rsc...@rs.com -Original Message- From: IBM Mainframe Discussion List [mailto:ibm-m...@bama.ua.edu] On Behalf Of Gerry Anstey Sent: 03 February 2009 11:25 To: IBM-MAIN@bama.ua.edu Subject: SDSF - so slow! OK bad design, we have lot of cack here, probably due to hiring cheap programmers, any way, I digress. Here s the SDSF summary: PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT JESMSGLG JES2 2 PSTGCS T LOCAL 52 JESJCL JES2 3 PSTGCS T LOCAL 44 JESYSMSG JES2 4 PSTGCS T LOCAL 94 DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M Now we had a need to extract some of the records in CMPRT01, I wrote job to run SDSF in batch and to use the PRINT ODSN command to extract the data to a data set. Then I read the dataset with Filemaster and extracted the desired records into a smaller file. My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? thanks Gerry Anstey Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
SDSF - so slow!
OK bad design, we have lot of cack here, probably due to hiring cheap programmers, any way, I digress. Here s the SDSF summary: PREFIX=GCSPROCP DEST=(ALL) OWNER=* SYSNAME=FMVS NP DDNAME STEPNAME PROCSTEP DSID OWNERC DEST REC-CNT JESMSGLG JES2 2 PSTGCS T LOCAL 52 JESJCL JES2 3 PSTGCS T LOCAL 44 JESYSMSG JES2 4 PSTGCS T LOCAL 94 DDPRINT GCSPROCP 106 PSTGCS 0 LOCAL 3 CMPRINT GCSPROCP 107 PSTGCS 0 LOCAL 28,559 CMPRT01 GCSPROCP 108 PSTGCS T LOCAL 25M Now we had a need to extract some of the records in CMPRT01, I wrote job to run SDSF in batch and to use the PRINT ODSN command to extract the data to a data set. Then I read the dataset with Filemaster and extracted the desired records into a smaller file. My questions are: 1. Any ideas why SDSF takes appox 90 minutes (12 EXCPs + 31mins CPU) to read and write out the data and Filemaster takes about 3 minutes to read 25million recs and write about 1.5million? 2. Is there any way to make SDSF extract faster? thanks Gerry Anstey Generally, this communication is for informational purposes only and it is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. In the event you are receiving the offering materials attached below related to your interest in hedge funds or private equity, this communication may be intended as an offer or solicitation for the purchase or sale of such fund(s). All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to UK legal entities. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@bama.ua.edu with the message: GET IBM-MAIN INFO Search the archives at http://bama.ua.edu/archives/ibm-main.html
