Re: Corrupted SMF record

[Shmuel Metz (Seymour J.)]

In <[EMAIL PROTECTED]>, on 05/14/2005
   at 10:05 AM, BarryMerrill <[EMAIL PROTECTED]> said:

>Historically, SMF records were created as VBS in the original OS/360
>implementation in 1964, because the OS developers only gave the SMF 
>component 1000 bytes of the nucleus for buffer space,
> (the PCP nucleus was only 89K total!)

Maybe yours[1] was, but OS/360 was designed for a 32KiB machine and
would IPL on a 64MiB machine. Even our MFT nucleus wasn't that big.

BTW, my recollection is that you needed MFT II or MVT for SMF.

[1] A lot of installation options affected the nucleus size.
 
-- 
 Shmuel (Seymour J.) Metz, SysProg and JOAT
 ISO position; see  
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Corrupted SMF record

[BarryMerrill]

Historically, SMF records were created as VBS in the original OS/360
implementation in 1964, because the OS developers only gave the SMF 
component 1000 bytes of the nucleus for buffer space,
 (the PCP nucleus was only 89K total!)
which the SMF team broke into two 500 byte buffers, and then used VBS,
as that was the only architecture that allowed you to write a logical
data record that contained 32756 bytes, when you only had 500 bytes
of real storage for your buffer.  The original DCB attributes
were RECFM=VBS,LRECL=32760,BLKSIZE=500!
   See my SHARE paper "The History of SMF", written in honor of the 
   Twentieth Anniversary of SMF in Newsletter FIFTEEN at
 http://www.mxg.com/Newsletters

While the "officially documented" LRECL/RDW in SMFxLEN in the
SMF header has has always been documented as 32756 bytes,
which is a maximum data length of only 32752, RDW=LRECL=32756.
many SMF record creators have read that as a data length of 32756,
and have created VBS records with LRECL=32760, and there have been
at least two instances in the past (SMF 79 subtype 1, TSO/MON) 
in which the actual data-length was 32760, which caused two physical
records to be created for that one logical record, one with the 
first 32756 bytes of data, and a second spanned record with the
remaining 4 bytes of the data in a second VBS record. 
Both of those records were corrected by their vendors when they 
were discovered.

Even recently, some new SMF records have been created with an
LRECL greater than 32760, and then corrected, otably  
APAR PQ7 corrects SMF type 119 (TCP/IP V3) records that had an
invalid BLKSIZE and LRECL of 32768.  The APAR text reconfirms the
SMF limit of 32756 bytes of data, i.e., LRECL=32760,BLKSIZE=32760.

Since QSAM has a hard limit of 32760 for both LRECL and BLKSIZE, 
the only absolutely safe DCB attributes to read an SMF VBS file
has always been RECFM=VBS,LRECL=32760,BLKSIZE=32760, which covers
records with either 32752, 35756, or 35760 bytes of real data.

But you cannot convert an SMF VBS file to VB, if it should contain
a record with RDW=LRECL=32760, i.e., with the full 32756 bytes of
data, because, unlike VBS, a VB file is required to have its RDW
value 4 bytes less than the BDW, and thus a VB file can only contain
a record with 32752 bytes of data, RDW=32756, and BLKSIZE=32760.
If you copy an LRECL=32760 VBS record to LRECL=32756 VB record,
you will loose the last four bytes of data.

Also, on DASD, VB with BLKSIZE=32760 will waste tremendous
space for those large blocks, while VBS can use BLKSIZE=0
(for half-track blocksize) to minimize space requirements.
Those half-track blocks will contain spanned records when
the LRECL exceeds the half-track blocksize.

 Okay, what about that LRECL=32767 instead of LRECL=32760 for VBS?
 The (stupid, IMO) default of IFASMFDP is LRECL=32767, although
 the records written to the VSAM file cannot exceed 32760?
 Yes, MVS currently does allow you to create a VBS file with 32767
 LRECL, but many other software products fail with 32767.
 I have sent numerous requests for IBM to change the illegitmate
 32767 back to the correct 32760, but the requests are unanswered.
 The z/OS 1.6 SMF manual for the dump program at least does state
   If you specify DCB parameters you must specify RECFM=VBS.
   You can specify LRECL=32760 instead of 32767.



A final note:  I always recommend the use of
 DCB=VBS,LRECL=32760,BLKSIZE=32760
to read any V, VB, or VBS file, since V and VB
are subsets of the VBS architecture when those
attributes are specified.  

In summary, you might well find that none of your SMF records from IBM
and other vendors exceed the data-length of 32752, LRECL of 32756, but
history suggests you'd place yourself at risk if you use those values.


Barry 
 

P.S.  SAS has always been able to handle broken VBS segments;
  in fact, after my August, 1974 presentation of how State
  Farm was using SAS to process SMF and that it would recover
  broken SMF data, a number of attendees purchased what was
  then the Statistical Analysis System, from the Statistics
  Department at North Carolina State University (for $100!!)
  solely to recover broken SMF data.
  Recall that with VBS data written to disk with a small 
  blocksize, any record with LRECL greater than BLKSIZE is
  spanned, so any system failure usually left you with a
  block with spanned data at the end, and then the next SMF
  block was the new IPL record after the failure, which
  caused IFASMFDP to die with an 002 ABEND because it cannot
  handle broken VBS segments!!


   DATA _NULL_;
INFILE SMF;
FILE SMFOUT DCB=SMF;
INPUT;
PUT _INFILE_;

  is all you need to recover that which is recoverable.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] w

Re: Corrupted SMF record

[Bruce Black]

Let me throw in an opinion that I have expressed before, but not for a 
few years.

When SMF records are dumped from the active SMF dataset, they can be 
converted to VB instead of VBS (spanned).  When I researched this some 
years ago I found that the SMF rules said that the largest valid SMF 
records was 32756 bytes, which is also the largest record allowed in 
VBS.  There were a few SMF bugs where records of 32760 or even 32767 
were created, but those were fixed.  I just checked the 1.6 SMF manual 
and it confirms that the max SMF record length is 32756 (search for 
"32.756" if you want to check).

I see that the description of the SMF dump program in the SMF manual 
still says that you can specify LRECL=32760 or 32767, even though the 
DCB description in the DFSMS manuals clearly states that the max is 
32752 plus 4 for the RDW (32756). 

So I have long advocated that SMF dump files should be VB to avoid all 
the issues with corrupted spanned records.  I even wrote an article in 
the old Technical Support magazine from Naspa, perhaps 10 years ago. 

As I recall, when I last raised this issue on this list, some people 
disputed me, and they may be right.  I have not reinvestigated this or 
tried to validate it since I wrote that article.   Perhaps some listers, 
with real-world SMF data, can scan their SMF files and see if they can 
determine the largest logical record on their data. 

BTW, for VBS SMF files on tape, our tape utility FATAR has support for 
identifying and deleting bad spanned records.

--
Bruce A. Black
Senior Software Developer for FDR
Innovation Data Processing
Little Falls, NJ 07424
973-890-7300
personal: [EMAIL PROTECTED]
sales info: [EMAIL PROTECTED]
tech support: [EMAIL PROTECTED]
web: www.fdr.innovationdp.com
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Corrupted SMF record

[McKown, John]

> -Original Message-
> From: IBM Mainframe Discussion List 
> [mailto:[EMAIL PROTECTED] On Behalf Of Laine, Rogers
> Sent: Friday, May 13, 2005 1:35 PM
> To: IBM-MAIN@BAMA.UA.EDU
> Subject: Corrupted SMF record
> 
> 
> Hello,
>  
> I have a SMF dataset that was MOD'ed to a existing dataset that is now
> corrupted using the program SMFDUMP.
> I was able to use the program IFASMFDP to create a new output 
> dataset up
> to the corrupted record.
> My question is how can I recover the records that are pass 
> the bad one?
> I recall seeing a work around on this list, any ideas? 
>  
> Rogers
>  

Use DFSORT.

//COPY EXEC PGM=SORT,REGION=0M
//SYSPRINT DD SYSOUT=*
//SORTMSG DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SORTIN  DD ...
//SORTOUT DD ...
//SYSIN DD *
 OPTION SPANINC=RC0
 SORT FIELDS=(COPY)
/*
//


--
John McKown
Senior Systems Programmer
UICI Insurance Center
Information Technology

This message (including any attachments) contains confidential
information intended for a specific individual and purpose, and its'
content is protected by law.  If you are not the intended recipient, you
should delete this message and are hereby notified that any disclosure,
copying, or distribution of this transmission, or taking any action
based on it, is strictly prohibited.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Corrupted SMF record

[Ed Gould]

on 5/13/05 1:35 PM, Laine, Rogers at [EMAIL PROTECTED] wrote:

> Hello,
> 
> I have a SMF dataset that was MOD'ed to a existing dataset that is now
> corrupted using the program SMFDUMP.
> I was able to use the program IFASMFDP to create a new output dataset up
> to the corrupted record.
> My question is how can I recover the records that are pass the bad one?
> I recall seeing a work around on this list, any ideas?
> 
> Rogers
> 

Rogerers,

Look at the CBTtape I think they have a S002 fixer upper.

Ed

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Re: Corrupted SMF record

[Chase, John]

> -Original Message-
> From: IBM Mainframe Discussion List On Behalf Of Laine, Rogers
> 
> Hello,
>  
> I have a SMF dataset that was MOD'ed to a existing dataset 
> that is now corrupted using the program SMFDUMP.
> I was able to use the program IFASMFDP to create a new output 
> dataset up to the corrupted record.
> My question is how can I recover the records that are pass 
> the bad one?
> I recall seeing a work around on this list, any ideas? 

Go to  and search for VBSFIX, which is (was?) a free
(hence unsupported) program that bypasses corrupted spanned records.

-jc-

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html

Corrupted SMF record

[Laine, Rogers]

Hello,
 
I have a SMF dataset that was MOD'ed to a existing dataset that is now
corrupted using the program SMFDUMP.
I was able to use the program IFASMFDP to create a new output dataset up
to the corrupted record.
My question is how can I recover the records that are pass the bad one?
I recall seeing a work around on this list, any ideas? 
 
Rogers
 
 



Confidentiality Notice:

This E-Mail transmission (and/or the documents accompanying it)
may contain information belonging to the sender which is 
confidential, privileged and/or exempt from disclosure under 
applicable law.  The information is intended only for the use
of the individual(s) or entity named above.   If you are not
the intended recipient, you are hereby  notified that any
disclosure, copying, distribution or the taking of any action
in reliance on the contents of this information is strictly 
prohibited.  If you have received this E-Mail transmission 
in error, please immediately notify us by return E-Mail or 
telephone to arrange for return of its contents including any
documents.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [EMAIL PROTECTED] with the message: GET IBM-MAIN INFO
Search the archives at http://bama.ua.edu/archives/ibm-main.html