Re: Transmitting SMF records

[Paul Gilmartin]

On Sat, 20 Jan 2024 00:14:58 -0500, Cheryl Watson wrote:

>I know that I’m late to this game, but we used to have a free tool to handle 
>this called ‘WWUNTERSE’, which has been used by hundreds of people. It is now 
>being made available by its original developer, Mario Bezzi, at this link - 
>https://www.ap4zlabs.com/free-tools. 
> 
What about 
?
Was this topic helpful?
Transmitting data sets
Last Updated: 2023-04-05
You can use the TRANSMIT command to transmit sequential or partitioned data 
sets with
record formats of F, FS, FB, FBS, V, VB, VBS, and U. The data sets must reside 
on a direct
access storage device (DASD). For a VB or VBS data set, the largest logical 
record length
(LRECL) TSO/E can transmit to VM is 65,535.  ...

>==
>On Wed, 25 Jan 2023 20:24:59 +, Andrew N Wilt wrote:
>
>>   ... Apparently, they are able to decipher the RDW records resulting from 
>> uploading a RECFM=VBS as a RECFM=U. Unfortunately, at this time, GDKUTIL 
>> doesn't have the smarts to parse the bytestream as RDW+data as it is writing 
>> to the output data set. That is a current Request For Enhancement, though.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Cheryl Watson]

I know that I’m late to this game, but we used to have a free tool to handle 
this called ‘WWUNTERSE’, which has been used by hundreds of people. It is now 
being made available by its original developer, Mario Bezzi, at this link - 
https://www.ap4zlabs.com/free-tools. 

All my best,
Cheryl Watson
==
Cheryl Watson Walker, CEO
Watson & Walker, Inc.
Sarasota, FL USA
www.watsonwalker.com
Cell/Text: 941-266-6609
==

On Jan 27, 2023, at 9:55 PM, Paul Gilmartin 
<042bfe9c879d-dmarc-requ...@listserv.ua.edu> wrote:

On Wed, 25 Jan 2023 20:24:59 +, Andrew N Wilt wrote:

>   ... Apparently, they are able to decipher the RDW records resulting from 
> uploading a RECFM=VBS as a RECFM=U. Unfortunately, at this time, GDKUTIL 
> doesn't have the smarts to parse the bytestream as RDW+data as it is writing 
> to the output data set. That is a current Request For Enhancement, though.
> 
It's not so hard.  The necessary format descriptions are in "Using Data Sets" (I
believe; perhaps also elsewhere.)  Long ago, I did an experiment; a PoC.  IBM
conveniently provided the input data in SMP/E SMPNTS which is exactly such
a bytestream representation of an IEBCOPY-unloaded data set, pax-compressed.
I uncompressed one with pax and allocated it as PATH(...) FILEDATA(BINARY)
RECFM(VB) LRECL(1000) ...  FILEDATA(BINARY) causes the BDW qnd SDW
images to be read as data (with EXECIO).  I was able to reorganize the 1000-byte
chunks into the original blocks.  In that era, EXECIO couldn't handle RECFM(U),
so I tried LMPUT.  First try  failed -- ISPF bug, fixed with APAR. (LMPUT had 
been
unable to deal with RECFM(U) with data above 16 MiB.  The surprising repair was
that LMPUT was changed to copy the data below the line.)  I took the output data
set, overrode RECFM(U) to RECFM=VBS and successfully reconstructed the
PDS with IEBCOPY.

Tthe RFE should provide a set of utilities to convert various DSORGs to simple
bytestreams and back.  They should be pipe-savvy, and FTP client should be made
pipe-savvy to eliminate requirements for temporary data sets.  Better sftp, 
which
is based on ssh, already pipe-savvy.  (But what about restart from broken
connection?)

All forbidden by Conway's Law.

> Sincerely,
> Andrew Wilt
> 
> DFSMSdfp Cloud Data Access Product Owner
> DFSMShsm development

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN







--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Wed, 25 Jan 2023 20:24:59 +, Andrew N Wilt wrote:

>... Apparently, they are able to decipher the RDW records resulting from 
> uploading a RECFM=VBS as a RECFM=U. Unfortunately, at this time, GDKUTIL 
> doesn't have the smarts to parse the bytestream as RDW+data as it is writing 
> to the output data set. That is a current Request For Enhancement, though.
>
It's not so hard.  The necessary format descriptions are in "Using Data Sets" (I
believe; perhaps also elsewhere.)  Long ago, I did an experiment; a PoC.  IBM
conveniently provided the input data in SMP/E SMPNTS which is exactly such
a bytestream representation of an IEBCOPY-unloaded data set, pax-compressed.
I uncompressed one with pax and allocated it as PATH(...) FILEDATA(BINARY)
RECFM(VB) LRECL(1000) ...  FILEDATA(BINARY) causes the BDW qnd SDW
images to be read as data (with EXECIO).  I was able to reorganize the 1000-byte
chunks into the original blocks.  In that era, EXECIO couldn't handle RECFM(U),
so I tried LMPUT.  First try  failed -- ISPF bug, fixed with APAR. (LMPUT had 
been
unable to deal with RECFM(U) with data above 16 MiB.  The surprising repair was
that LMPUT was changed to copy the data below the line.)  I took the output data
set, overrode RECFM(U) to RECFM=VBS and successfully reconstructed the
PDS with IEBCOPY.

Tthe RFE should provide a set of utilities to convert various DSORGs to simple
bytestreams and back.  They should be pipe-savvy, and FTP client should be made
pipe-savvy to eliminate requirements for temporary data sets.  Better sftp, 
which
is based on ssh, already pipe-savvy.  (But what about restart from broken
connection?)

All forbidden by Conway's Law.

>Sincerely,
>Andrew Wilt
>
>DFSMSdfp Cloud Data Access Product Owner
>DFSMShsm development

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Michael Oujesky]

And just an FYI, but the logical record lengths SAS reads can be up 
to 16,777,215.


Michael

At 05:40 PM 1/25/2023, Kirk Wolf wrote:
ISTR that distributed SAS has an option to read binary byte stream 
files with BDW+RDWs, which is what you would get if the program were 
to produce a merged RECFM=U DCB.


Kirk Wolf
Dovetailed Technologies, LLC
http://coztoolkit.com
Dovetailed Technologies: +1 636.300.0901

Note: Our website and domain name have changed from dovetail.com to 
coztoolkit.com


On Wed, Jan 25, 2023, at 4:23 PM, Andrew Rowley wrote:
> On 26/01/2023 7:24 am, Andrew N Wilt wrote:
>
> > The GDKUTIL utility does allow an Upload of something with 
RECFM=U. This was done at the behest of a customer that was using 
ftp to put SMF data out for processing by the SAS tool. Apparently, 
they are able to decipher the RDW records resulting from uploading 
a RECFM=VBS as a RECFM=U.

>
> I would be slightly surprised if SAS can't read VB records properly
> formatted with the RDW. RECFM U was a workaround for the problem where
> FTP stripped out the RDW. So GDKUTIL seems to be reproducing the
> brokenness of FTP, then providing the same workaround as a feature.
>
> It would be preferable if it retained the RDW in VB records by default.
> GDKUTIL seems new enough that the default could be changed without too
> much impact. I doubt there is anyone using variable length binary
> transfers currently who needs the data without the RDW. (However there
> might be people uploading data who haven't discovered the resulting data
> is unusable.)
>
> I have been working with SMF data myself for many years, and this
> problem with FTP causes a LOT of customer confusion and frustration.
>
> --
> Andrew Rowley
> Black Hill Software
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 26/01/2023 10:40 am, Kirk Wolf wrote:

ISTR that distributed SAS has an option to read binary byte stream files with 
BDW+RDWs, which is what you would get if the program were to produce a merged 
RECFM=U DCB.


It does, but I can't see any reason to implement that other than to 
workaround IBM's broken FTP. It's easier to read data in record format 
with RDWs.


IBM FTP will do the right thing if you specify the SITE RDW option - I 
assume that the SAS RECFM U support was added before that was available.


EasySMF will automatically recognize RECFM=U, RECFM=V or TSO TRANSMIT 
format data, optionally in a gzip or zip file, and process it correctly. 
It will also recognize SMF data with no RDW and tell you what is wrong.


--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 26/01/2023 10:20 am, Paul Gilmartin wrote:
It's possibly catering to the novice user rather than the journeyman. 


Unfortunately I don't think that's the case. When transferring variable 
length binary records, the default options give you unusable (I would 
call it corrupted) data. You need to delve into the details of record 
formats and RDWs to understand why.


If using the data requires deblocking variable records from RECFM U 
blocks it's not for the novice user. More likely, the people writing FTP 
didn't properly understand the requirements for variable length records. 
You need the RDW.


--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[David Spiegel]

Hi Paul,
Do you DFSMSdss->AMATERSE->XMIT large amounts of SMF Data? (Probably not.)
I would never try to XMIT a Dataset like this (assuming that you could 
even have enough space for the XMIT Output.)
Not only that , but BLKSIZE=3120 is inefficient and really slow (and 
with no Transmit Exit, generate a huge amount messages to the TSO User).

You're much better off FTPing the AMATERSE Output.

Regards,
David

On 2023-01-25 18:31, Paul Gorlinsky wrote:
>From my experience with one of the vendors, NOT SAS, all of these products just expect a stream of data to process. If you are using PC versions, just download as binary ... the VBS data. The import programs parse the data correctly.   Going from Mainframe to Mainframe, save yourself a lot of headache and use ADRDSSU DUMP to create a Disk File of the data, Terse ( PACK ) it, and using TRANSMIT across NJE or TRANSMIT to a data set that is RECFM FB LRECL 80 BLKSIZE 0 ( 3120 ) and FTP the TRANSMIT DSN as binary, recfm fb, LRECL 80... and RECEIVE, deterse (unpack ) and ADRDSSU RESTORE ... 


We do this all the time with VSAM, QSAM, PDS, PDSE,  data... even sometimes 
but a PC in between ...

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Kirk Wolf]

ISTR that distributed SAS has an option to read binary byte stream files with 
BDW+RDWs, which is what you would get if the program were to produce a merged 
RECFM=U DCB.

Kirk Wolf
Dovetailed Technologies, LLC
http://coztoolkit.com
Dovetailed Technologies: +1 636.300.0901

Note: Our website and domain name have changed from dovetail.com to 
coztoolkit.com

On Wed, Jan 25, 2023, at 4:23 PM, Andrew Rowley wrote:
> On 26/01/2023 7:24 am, Andrew N Wilt wrote:
> 
> > The GDKUTIL utility does allow an Upload of something with RECFM=U. This 
> > was done at the behest of a customer that was using ftp to put SMF data out 
> > for processing by the SAS tool. Apparently, they are able to decipher the 
> > RDW records resulting from uploading a RECFM=VBS as a RECFM=U.
> 
> I would be slightly surprised if SAS can't read VB records properly 
> formatted with the RDW. RECFM U was a workaround for the problem where 
> FTP stripped out the RDW. So GDKUTIL seems to be reproducing the 
> brokenness of FTP, then providing the same workaround as a feature.
> 
> It would be preferable if it retained the RDW in VB records by default. 
> GDKUTIL seems new enough that the default could be changed without too 
> much impact. I doubt there is anyone using variable length binary 
> transfers currently who needs the data without the RDW. (However there 
> might be people uploading data who haven't discovered the resulting data 
> is unusable.)
> 
> I have been working with SMF data myself for many years, and this 
> problem with FTP causes a LOT of customer confusion and frustration.
> 
> -- 
> Andrew Rowley
> Black Hill Software
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Steve Beaver]

Terse the file and be done with it

Sent from my iPhone

No one said I could type with one thumb 

> On Jan 25, 2023, at 17:31, Paul Gorlinsky  wrote:
> 
> From my experience with one of the vendors, NOT SAS, all of these products 
> just expect a stream of data to process. If you are using PC versions, just 
> download as binary ... the VBS data. The import programs parse the data 
> correctly.   Going from Mainframe to Mainframe, save yourself a lot of 
> headache and use ADRDSSU DUMP to create a Disk File of the data, Terse ( PACK 
> ) it, and using TRANSMIT across NJE or TRANSMIT to a data set that is RECFM 
> FB LRECL 80 BLKSIZE 0 ( 3120 ) and FTP the TRANSMIT DSN as binary, recfm fb, 
> LRECL 80... and RECEIVE, deterse (unpack ) and ADRDSSU RESTORE ... 
> 
> We do this all the time with VSAM, QSAM, PDS, PDSE,  data... even 
> sometimes but a PC in between ... 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gorlinsky]

From my experience with one of the vendors, NOT SAS, all of these products just 
expect a stream of data to process. If you are using PC versions, just download 
as binary ... the VBS data. The import programs parse the data correctly.   
Going from Mainframe to Mainframe, save yourself a lot of headache and use 
ADRDSSU DUMP to create a Disk File of the data, Terse ( PACK ) it, and using 
TRANSMIT across NJE or TRANSMIT to a data set that is RECFM FB LRECL 80 BLKSIZE 
0 ( 3120 ) and FTP the TRANSMIT DSN as binary, recfm fb, LRECL 80... and 
RECEIVE, deterse (unpack ) and ADRDSSU RESTORE ... 

We do this all the time with VSAM, QSAM, PDS, PDSE,  data... even sometimes 
but a PC in between ... 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Thu, 26 Jan 2023 09:23:54 +1100, Andrew Rowley wrote:

>On 26/01/2023 7:24 am, Andrew N Wilt wrote:
>...
>I would be slightly surprised if SAS can't read VB records properly
>formatted with the RDW. RECFM U was a workaround for the problem where
>FTP stripped out the RDW. So GDKUTIL seems to be reproducing the
>brokenness of FTP, then providing the same workaround as a feature.
>
At times, FTP is too damned smart.  I have allocated a DDNAME with overriding
RECFM=U and attempted to transfer from //DD:...  But it failed.  It appeared 
that
FTP did a DYNALLOC INFO, found the attributes in the DSCB, and ignored those
I specified.  It's possibly catering to the novice user rather than the 
journeyman.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 26/01/2023 7:24 am, Andrew N Wilt wrote:


The GDKUTIL utility does allow an Upload of something with RECFM=U. 
This was done at the behest of a customer that was using ftp to put SMF data 
out for processing by the SAS tool. Apparently, they are able to decipher the 
RDW records resulting from uploading a RECFM=VBS as a RECFM=U.


I would be slightly surprised if SAS can't read VB records properly 
formatted with the RDW. RECFM U was a workaround for the problem where 
FTP stripped out the RDW. So GDKUTIL seems to be reproducing the 
brokenness of FTP, then providing the same workaround as a feature.


It would be preferable if it retained the RDW in VB records by default. 
GDKUTIL seems new enough that the default could be changed without too 
much impact. I doubt there is anyone using variable length binary 
transfers currently who needs the data without the RDW. (However there 
might be people uploading data who haven't discovered the resulting data 
is unusable.)


I have been working with SMF data myself for many years, and this 
problem with FTP causes a LOT of customer confusion and frustration.


--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Wed, 25 Jan 2023 20:24:59 +, Andrew N Wilt wrote:
>...
>   The GDKUTIL utility does allow an Upload of something with RECFM=U. 
> This was done at the behest of a customer that was using ftp to put SMF data 
> out for processing by the SAS tool. Apparently, they are able to decipher the 
> RDW records resulting from uploading a RECFM=VBS as a RECFM=U. Unfortunately, 
> at this time, GDKUTIL doesn't have the smarts to parse the bytestream as 
> RDW+data as it is writing to the output data set. That is a current Request 
> For Enhancement, though.
>
I've done that sort of thing with REXX.  A small REXX utility, perhaps in 
SAMPLIB,
to do that transformation would serve the purpose and might have more general
use, as for files transmitted with FTP overriding to RECFM=U.

Modularity is valuable.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew N Wilt]

Hi Andrew,
You have a good point. The GDKUTIL utility is pretty new to z/OS, and 
we were going for meeting the customer requirement. That was specifically to 
provide a JCL utility that could download the sequential data in a Cloud Object 
to z/OS, as well as upload z/OS data to a Cloud Object with the intent that the 
data was usable by applications running in the Cloud. I specifically added that 
note to the documentation so that no one was surprised by the behavior when 
targeting a z/OS data set.

We are actively updating the functionality of the utility and would 
love some customer-driven direction. So, please open an RFE (or now Aha! Idea) 
against the cloud_data_access component under z/OS with whatever ideas or 
wishes you have.

The GDKUTIL utility does allow an Upload of something with RECFM=U. 
This was done at the behest of a customer that was using ftp to put SMF data 
out for processing by the SAS tool. Apparently, they are able to decipher the 
RDW records resulting from uploading a RECFM=VBS as a RECFM=U. Unfortunately, 
at this time, GDKUTIL doesn't have the smarts to parse the bytestream as 
RDW+data as it is writing to the output data set. That is a current Request For 
Enhancement, though.

Sincerely,
Andrew Wilt

DFSMSdfp Cloud Data Access Product Owner
DFSMShsm development

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Andrew Rowley
Sent: Monday, January 23, 2023 3:55 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Transmitting SMF records

On 20/01/2023 7:07 am, Glenn Wilcock wrote:
> Late to the party on this discussion... DFSMS recently shipped a new utility 
> named GDKUTIL.  It converts z/OS data sets and UNIX files to cloud objects 
> and visa-versa.

Reading the documentation, the following note looks like a problem:

=

/When specifying a z/OS Data Set for LOCAL or LOCNAME for a DOWNLOAD command 
without the CONVERT keyword, the Record Format should not be Variable. (RECFM=V 
or RECFM=VB) This is because in binary mode, there are no indicators in the 
data where a record ends, so the data is placed up to the maximum logical 
record length. If a Data Set with variable records was uploaded to a Cloud 
Object in binary mode, the indicators of where a record begins and ends are 
lost. Therefore, a download operation of that same data to a Variable record 
data set cannot tell when to start a new record in the data set./

=

A useful function should be able to round trip the data to and from cloud 
storage and end up with exactly what you started with.

If there is not enough information to recreate the records on z/OS with the 
correct length, there is probably not enough information to usefully process 
the data on another platform. FTP made the same mistake, assuming that the 
record length information was not part of the data. 
For variable length records it is critical.

RECFM U presumably preserves the block & record length information, but makes 
it unnecessarily complex to decipher. From the documentation I can't tell 
whether uploading RECFM U data results in the same data you started with. At a 
minimum there must be complications to end up with the real/correct DCB 
attributes.

-- 
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 20/01/2023 7:07 am, Glenn Wilcock wrote:

Late to the party on this discussion... DFSMS recently shipped a new utility 
named GDKUTIL.  It converts z/OS data sets and UNIX files to cloud objects and 
visa-versa.


Reading the documentation, the following note looks like a problem:

=

/When specifying a z/OS Data Set for LOCAL or LOCNAME for a DOWNLOAD 
command without the CONVERT keyword, the Record Format should not be 
Variable. (RECFM=V or RECFM=VB) This is because in binary mode, there 
are no indicators in the data where a record ends, so the data is placed 
up to the maximum logical record length. If a Data Set with variable 
records was uploaded to a Cloud Object in binary mode, the indicators of 
where a record begins and ends are lost. Therefore, a download operation 
of that same data to a Variable record data set cannot tell when to 
start a new record in the data set./


=

A useful function should be able to round trip the data to and from 
cloud storage and end up with exactly what you started with.


If there is not enough information to recreate the records on z/OS with 
the correct length, there is probably not enough information to usefully 
process the data on another platform. FTP made the same mistake, 
assuming that the record length information was not part of the data. 
For variable length records it is critical.


RECFM U presumably preserves the block & record length information, but 
makes it unnecessarily complex to decipher. From the documentation I 
can't tell whether uploading RECFM U data results in the same data you 
started with. At a minimum there must be complications to end up with 
the real/correct DCB attributes.


--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew N Wilt]

Hi Michael,
Unfortunately, the online publications haven't been updated yet. I 
believe you will see the changes in the next major update of the manuals in 8 
months or so. Knowing it would take a while, I put the pdf of the changes out 
on the web with the APAR. 

If you have any questions, feel free to contact me directly. 
(anw...@us.ibm.com)

Sincerely,
Andrew Wilt

DFSMSdfp Cloud Data Access Product Owner
DFSMShsm development

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Michael Watkins
Sent: Friday, January 20, 2023 11:27 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Transmitting SMF records

Glenn Wilcock shared 
https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA62318/Cloud_Object_Utility_Pub_updates.pdf

The document at this URL states: z/OS MVS Programming: Callable Services for 
High Level Languages is updated to add a new chapter after 'Part 10: SMF 
Services' as follows: 'Part 11: Cloud Data Access Services'.

However, when I use either google or an IBM URL 
(https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R5Library?OpenDocument)
 to search for this document, I only find an earlier version without the 
additional Part 11.

Can anyone provide a newer link to IBM documentation where I can find this 
updated version of the SA23-1377 document?

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Glenn Wilcock
Sent: Thursday, January 19, 2023 2:07 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

CAUTION: This email originated from outside of the Texas Comptroller's email 
system.
DO NOT click links or open attachments unless you expect them from the sender 
and know the content is safe.

Late to the party on this discussion... DFSMS recently shipped a new utility 
named GDKUTIL.  It converts z/OS data sets and UNIX files to cloud objects and 
visa-versa.  This enables utilizing cloud object storage as a method for z/OS 
sharing data, such as SMF, as opposed to FTPing.  Use GDKUTIL to write the data 
to object storage for shared access across sysplexes / platforms and visa-versa 
- ingest off-platform data into z/OS.  DFSMS CDA (Cloud Data Access) is used to 
securely manage cloud access.

https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpublic.dhe.ibm.com%2Feserver%2Fzseries%2Fzos%2FDFSMS%2FCDA%2FOA62318%2FCloud_Object_Utility_Pub_updates.pdf=05%7C01%7Cmichael.watkins%40CPA.TEXAS.GOV%7C22dc17e633fc4131b2d908dafa58c9cd%7C2055feba299d4d0daa5a73b8b42fef08%7C0%7C0%7C638097556513326232%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=grotivTONiSwuDBIiUJdl6Pb7uZAcsgMkWFgAgIzyXk%3D=0
 

Glenn Wilcock, DFSMS Chief Product Owner

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Michael Watkins]

Glenn Wilcock shared 
https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA62318/Cloud_Object_Utility_Pub_updates.pdf

The document at this URL states: z/OS MVS Programming: Callable Services for 
High Level Languages is updated to add a new chapter after 'Part 10: SMF 
Services' as follows: 'Part 11: Cloud Data Access Services'.

However, when I use either google or an IBM URL 
(https://www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R5Library?OpenDocument)
 to search for this document, I only find an earlier version without the 
additional Part 11.

Can anyone provide a newer link to IBM documentation where I can find this 
updated version of the SA23-1377 document?

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Glenn Wilcock
Sent: Thursday, January 19, 2023 2:07 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

CAUTION: This email originated from outside of the Texas Comptroller's email 
system.
DO NOT click links or open attachments unless you expect them from the sender 
and know the content is safe.

Late to the party on this discussion... DFSMS recently shipped a new utility 
named GDKUTIL.  It converts z/OS data sets and UNIX files to cloud objects and 
visa-versa.  This enables utilizing cloud object storage as a method for z/OS 
sharing data, such as SMF, as opposed to FTPing.  Use GDKUTIL to write the data 
to object storage for shared access across sysplexes / platforms and visa-versa 
- ingest off-platform data into z/OS.  DFSMS CDA (Cloud Data Access) is used to 
securely manage cloud access.

https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpublic.dhe.ibm.com%2Feserver%2Fzseries%2Fzos%2FDFSMS%2FCDA%2FOA62318%2FCloud_Object_Utility_Pub_updates.pdf=05%7C01%7Cmichael.watkins%40CPA.TEXAS.GOV%7C22dc17e633fc4131b2d908dafa58c9cd%7C2055feba299d4d0daa5a73b8b42fef08%7C0%7C0%7C638097556513326232%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=grotivTONiSwuDBIiUJdl6Pb7uZAcsgMkWFgAgIzyXk%3D=0

Glenn Wilcock, DFSMS Chief Product Owner

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Glenn Wilcock]

Late to the party on this discussion... DFSMS recently shipped a new utility 
named GDKUTIL.  It converts z/OS data sets and UNIX files to cloud objects and 
visa-versa.  This enables utilizing cloud object storage as a method for z/OS 
sharing data, such as SMF, as opposed to FTPing.  Use GDKUTIL to write the data 
to object storage for shared access across sysplexes / platforms and visa-versa 
- ingest off-platform data into z/OS.  DFSMS CDA (Cloud Data Access) is used to 
securely manage cloud access.

https://public.dhe.ibm.com/eserver/zseries/zos/DFSMS/CDA/OA62318/Cloud_Object_Utility_Pub_updates.pdf

Glenn Wilcock, DFSMS Chief Product Owner

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Seymour J Metz]

Rackspace has historically had reputation issues and has found itself on 
anti-abuse block lists. It would be advisable to check whether any of your 
Rackspace IP addresses are listed.


From: IBM Mainframe Discussion List  on behalf of 
Boesel Guillaume 
Sent: Thursday, December 15, 2022 11:02 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

>From Barry Merrill :


My mxg.com email server company, RACKSPACE, took a Ransomeware attack and I'm 
unable to originate mail from ba...@mxg.com, although I am receiving forwards.
Can you post this to the IBM-MAIN track for me.

The problem with moving SMF records with ftp is that whenever ftp sees 
RECFM=V/VB/VBS,
it strips the BDW and RDW fields and only sends the data in those records, 
which is a totally
useless file.

Terse works,  for z/OS to z/OS, and also works sending to ASCII, where you can 
using the UNTERSE.EXE program that Cheryl Watson provides. to recreate the 
original VBS file.

But if the destination is for ASCII and SAS, you can use IEBGENER to create a 
copy of
the data, on z/OS, but using RECFM=U, which ftp can't muck-up, and SAS on ASCII 
processes that data using RECFM=S370VBS, since the file has the BDW and RDW, so 
the downloaded file
RECFM=U file can be read directly by SAS.

 // EXEC PGM=IEBGENER
 //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
 //SYSUT2 DD DSN=YOUR.NEW.RECFMU,DISP=(,CATLG),RECFM=U,
 //  BLKSIZE=32760,UNIT=SYSDA,SPACE=(CYL,(50,50))
 //SYSIN  DD DUMMY

And since the RECFM=U file has the BDW and RDW, it could be processed on z/OS
to recreate the original VBS data.

Merrilly Christmas,
Barry Merrill

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Scott Chapman]

Ah, I missed that or forgot by the time I got to posting. I haven't tried it 
myself, but have heard it is problematic to get the data back into a z/OS 
dataset in a usable fashion. 

On Sat, 17 Dec 2022 09:47:39 +1100, Andrew Rowley 
 wrote:

>The OP specified being able to reverse the process, so my understanding
>was it needed to transfer back to a dataset on z/OS.
>
>This seems to be surprisingly difficult - IBM doesn't seem to have
>considered round trip capability when they wrote the FTP functions.
>(Although I haven't tried it with a RECFM U transfer.)
>
>--
>Andrew Rowley
>Black Hill Software
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Fri, 16 Dec 2022 15:03:54 -0600, Kirk Wolf wrote:

>The OP didn't specify IBM FTP, so I'll mention that could use Co:Z SFTP, which 
>has a subcomment "dsput" which will allocate a dataset on the target system 
>matching the source system and then upload the dataset in binary, preserving 
>record formats.
>...
>Note: this works for RECFM=VBS with LRECL <= 32760, which is the range allowed 
>by dynamic allocation.   If you don't specify a DCB LRECL on your IFASMFDP 
>job, you will get RECFM=VBS with the magic LRECL=32767 (LRECL=X). 
>
Why not respect the attributes in the DSCB, which are merged into the DCB by 
the OPEN macro?
Or are you relying on a technique similar to IEBCOPY which represents a PDS 
block in the PDSU
as a logical record in a RECFM=VBS data set.

>... This is IMO a bug, since the maximum record length for SMF is 32756.   
> So put DCB=(RECFM=VBS,LRECL=32760) and stay away from that nonsense.
>
I believe that LRECL=X is represented as '8000'x (32768).  I learned this on 
one of these fora;
I nave not found it in any Ref. (But I haven't scrutinized all the Data Areas 
manuals.)  In any
case,  BPXWDYN accepts 32768 and creates a LRECL=X data set.

(The BPXWDYN designer may have been influenced by Doug Gwyn's maxim:
.)

And the envelope should contain a representation of the attributes so they would
be restored automatically on the return trip.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Sat, 17 Dec 2022 09:47:39 +1100, Andrew Rowley wrote:
>
>This seems to be surprisingly difficult - IBM doesn't seem to have
>considered round trip capability when they wrote the FTP functions.
>(Although I haven't tried it with a RECFM U transfer.)
> 
Surprisingly difficult or predictably IBM?

IBM didn't consider; made at best a half-hearted and didn't adequately test.

Even the combination:
BINARY
RDW
doesn't resunt in a round trip with integrity for a simple RECFM=VB data set.

And the envelope should contain a representation of the attributes so they would
be restored automatically on the return trip.

Does AMATERSE do this?  Does ADRDSSU?  If so, one of those should be made
an option supported by FTP.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 16/12/2022 11:33 pm, Scott Chapman wrote:


We have customers (S)FTP(S)ing us data every day directly referencing the 
output of IFASMFDP with DCB=RECFM=U with no problem.


The OP specified being able to reverse the process, so my understanding 
was it needed to transfer back to a dataset on z/OS.


This seems to be surprisingly difficult - IBM doesn't seem to have 
considered round trip capability when they wrote the FTP functions. 
(Although I haven't tried it with a RECFM U transfer.)


--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Kirk Wolf]

The OP didn't specify IBM FTP, so I'll mention that could use Co:Z SFTP, which 
has a subcomment "dsput" which will allocate a dataset on the target system 
matching the source system and then upload the dataset in binary, preserving 
record formats.

Example:

//SFTP EXEC PROC=SFTPPROC 
//SFTPIN DD * 
pwdsn="COZUSER.COZ.SAMPJCL(PW)" 
user=myuser
host=myhost 

. $script_dir/sftp_connect.sh <http://coztoolkit.com
Dovetailed Technologies: +1 636.300.0901

Note: Our website and domain name have changed from dovetail.com to 
coztoolkit.com

On Thu, Dec 15, 2022, at 8:43 AM, Gary Weinhold wrote:
> I'm glad there was a solution.
> 
> But the underlying problem seems to be that z/OS FTP appears to be limited to 
> processing a binary (image) transfer as a stream of bytes unless the transfer 
> is between a z/OS client and z/OS server.  IBM RECFM of VB and VBS seem to be 
> quite well defined and I don't know why, if the receiving z/OS FTP is 
> informed that the data stream is actually VB(S) format, either by 
> SITE/LOCSITE FTP commands or by the DCB of a preallocated output dataset, it 
> can't invoke the same routines that it would if the sender was a z/OS system.
> 
> As an alternative, it wouldn't seem very difficult to create a utility to 
> read the FTPed data (received as a sequential file with an arbitrary record 
> length) and reformat the data to write a VB(S) file.  I've known of 
> windows-based utilities that process FTPed SMF data (raw or tersed) so the 
> technical knowledge is out there.
> Date:Wed, 14 Dec 2022 13:22:27 -0600
> From:    Boesel Guillaume 
> Subject: Re: [EXTERNAL] Re: Transmitting SMF records
> 
> Hi Rex,
> Great. You are right, tersing file from tape to tape works well.
> It took around 80-90 MSU during an hour for just one file but it worked.
> 
> Hoping that Ituriel will be able to read this file.
> 
> Regards and thanks !
> 
> 
> 
> 
> Gary Weinhold
> Senior Application Architect
> DATAKINETICS | Data Performance & Optimization
> Phone:+1.613.523.5500 x216
> Email: weinh...@dkl.com
> Visit us online at www.DKL.com <http://www.dkl.com/>
> E-mail Notification: The information contained in this email and any 
> attachments is confidential and may be subject to copyright or other 
> intellectual property protection. If you are not the intended recipient, you 
> are not authorized to use or disclose this information, and we request that 
> you notify us by reply mail or telephone and delete the original message from 
> your mail system.
> 
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Scott Chapman]

We have customers (S)FTP(S)ing us data every day directly referencing the 
output of IFASMFDP with DCB=RECFM=U with no problem. From our standard 
instructions:

//SENDFTP EXEC PGM=FTP,PARM='(EXIT=12'
//SYSPRINT DD SYSOUT=*
//OUTPUT DD SYSOUT=*
//SMFFILEA DD DSN=*.SMFDMP.DUMPOUT,DCB=RECFM=U,BLKSIZE=32760,
// DISP=(OLD,DELETE) 
//SYSIN DD *
ftp.pivotor.com userid password
BINARY
LOCSITE EPSV4
QUOTE EPSV 
PUT //DD:SMFFILEA Company.Dyymmdd.SysplexName.SystemName.SMFU

BDWs and RDWs come through fine. (I think "quote epsv" is extraneous but it's 
been in the instructions forever. LOCSITE EPSV4 can be important for getting 
through some firewalls properly.)

Scott Chapman 

On Thu, 15 Dec 2022 12:27:56 -0600, Paul Gilmartin  wrote:

>>From Barry Merrill :
>>...
>>But if the destination is for ASCII and SAS, you can use IEBGENER to create a 
>>copy of
>>the data, on z/OS, but using RECFM=U, which ftp can't muck-up, and SAS on 
>>ASCII processes that data using RECFM=S370VBS, since the file has the BDW and 
>>RDW, so the downloaded file
>>RECFM=U file can be read directly by SAS.
>>
>> // EXEC PGM=IEBGENER
>> //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
>> ...
>I have tried to shortcut such a process at this point by:
> // EXEC PGM=FTP
> //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
> //INPUR DD *
> ...
> binary
> put DD:SYSUT1 ...
> ...
>Only to be dismayed that FTP apparently reads the DSCB and lets that dominate
>attributes coded on the DD statement.
>
>This is a significant transgression of MVS conventions.
>
>-- 
>gil
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Andrew Rowley]

On 16/12/2022 1:43 am, Gary Weinhold wrote:

As an alternative, it wouldn't seem very difficult to create a utility to read 
the FTPed data (received as a sequential file with an arbitrary record length) 
and reformat the data to write a VB(S) file.


It is not difficult. The biggest problem is that by default FTP strips 
the RDW, which makes variable length binary data unusable. The solution 
to that problem is to use the SITE RDW option for the download.


I wrote a Java program to read/write z/OS data in GZIP format to get 
around the various problems.


You can:

- gzip VB(S) into a FB dataset on z/OS, download the data and unzip on a PC

- download VB(S) data using BINARY and SITE RDW options, gzip the data 
on the PC, upload to a FB dataset then uncompress to a VB(S) dataset.


The program is available here:

https://github.com/BlackHillSoftware/z-java/tree/master/java/compress

--
Andrew Rowley
Black Hill Software

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

>From Barry Merrill :
>...
>But if the destination is for ASCII and SAS, you can use IEBGENER to create a 
>copy of
>the data, on z/OS, but using RECFM=U, which ftp can't muck-up, and SAS on 
>ASCII processes that data using RECFM=S370VBS, since the file has the BDW and 
>RDW, so the downloaded file
>RECFM=U file can be read directly by SAS.
>
> // EXEC PGM=IEBGENER
> //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
> ...
I have tried to shortcut such a process at this point by:
 // EXEC PGM=FTP
 //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
 //INPUR DD *
 ...
 binary
 put DD:SYSUT1 ...
 ...
Only to be dismayed that FTP apparently reads the DSCB and lets that dominate
attributes coded on the DD statement.

This is a significant transgression of MVS conventions.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Thu, 15 Dec 2022 15:31:31 +, Seymour J Metz  wrote:

>There are several facilities for encoding variable-length recrds into FB, 
>e.g., XMIT. A bit kludgy, but it works.
>
Alas, one needs to operate at both the sending and receiving ends because
z/OS sort lacks anything such as a QUOTE SITE XMIT mode.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Boesel Guillaume]

From Barry Merrill :


My mxg.com email server company, RACKSPACE, took a Ransomeware attack and I'm 
unable to originate mail from ba...@mxg.com, although I am receiving forwards.
Can you post this to the IBM-MAIN track for me.

The problem with moving SMF records with ftp is that whenever ftp sees 
RECFM=V/VB/VBS,
it strips the BDW and RDW fields and only sends the data in those records, 
which is a totally
useless file.

Terse works,  for z/OS to z/OS, and also works sending to ASCII, where you can 
using the UNTERSE.EXE program that Cheryl Watson provides. to recreate the 
original VBS file.

But if the destination is for ASCII and SAS, you can use IEBGENER to create a 
copy of
the data, on z/OS, but using RECFM=U, which ftp can't muck-up, and SAS on ASCII 
processes that data using RECFM=S370VBS, since the file has the BDW and RDW, so 
the downloaded file
RECFM=U file can be read directly by SAS.

 // EXEC PGM=IEBGENER
 //SYSUT1 DD DSN=YOUR.VB.FILE,DISP=SHR,RECFM=U,BLKSIZE=32760
 //SYSUT2 DD DSN=YOUR.NEW.RECFMU,DISP=(,CATLG),RECFM=U,
 //  BLKSIZE=32760,UNIT=SYSDA,SPACE=(CYL,(50,50))
 //SYSIN  DD DUMMY

And since the RECFM=U file has the BDW and RDW, it could be processed on z/OS
to recreate the original VBS data.

Merrilly Christmas,
Barry Merrill

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Thu, 15 Dec 2022 15:24:04 +, Pommier, Rex  wrote:

>What I remember from a discussion long ago was that the reason a VBS dataset 
>can't be put back together again is that when the VBS dataset is sent to a 
>Windows intermediary, some of the binary metadata defining the records and 
>blocks is lost by the Windows side, making it impossible to recreate the VBS 
>on the other end.  Whether this is actually true or not, that's what I recall. 
> Going straight z/OS to z/OS that metadata is retained.  
> 
In another experiment with the RDW option a get from MVS to desktop put RDW
images in the data stream.  But a put back to MVS resulted in double RDWs in
the MVS data set.  It appears that coders misunderstood the intent of the
designers.

And MVS-to-MVS with STRU R caused segments of a VBS data set to be
broken into individual records.

IBM could have done better.  Any one else would have done better.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Thu, 15 Dec 2022 14:43:48 +, Gary Weinhold wrote:

>I'm glad there was a solution.
>
>But the underlying problem seems to be that z/OS FTP appears to be limited to 
>processing a binary (image) transfer as a stream of bytes unless the transfer 
>is between a z/OS client and z/OS server.  ...
>
Not entirely true.  In days of yore, from an MVS server and a desktop client,
I have done such as:

binary
quote site type e
quote site mode b

MVS encapsulates the data by its conventions and the desktop sees a
simple "stream of  bytes".  Reversing the process with a "put" with a diffenent
MVS server and the desktop client correctly reconstitutes the data.  Or, a
desktop filter could be written to decode the data stream.

Likewise, the WUSTL FTP server performed conversions on-the-fly, such
as zipping a directory.

z/OS could do better than it does, such as AMATERSE on-the-fly with a SITE
command.

Do the Dovetailed or Rocket products do something similar?

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Seymour J Metz]

There are several facilities for encoding variable-length recrds into FB, e.g., 
XMIT. A bit kludgy, but it works.


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3


From: IBM Mainframe Discussion List [IBM-MAIN@LISTSERV.UA.EDU] on behalf of 
Gary Weinhold [weinh...@dkl.com]
Sent: Thursday, December 15, 2022 9:43 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

I'm glad there was a solution.

But the underlying problem seems to be that z/OS FTP appears to be limited to 
processing a binary (image) transfer as a stream of bytes unless the transfer 
is between a z/OS client and z/OS server.  IBM RECFM of VB and VBS seem to be 
quite well defined and I don't know why, if the receiving z/OS FTP is informed 
that the data stream is actually VB(S) format, either by SITE/LOCSITE FTP 
commands or by the DCB of a preallocated output dataset, it can't invoke the 
same routines that it would if the sender was a z/OS system.

As an alternative, it wouldn't seem very difficult to create a utility to read 
the FTPed data (received as a sequential file with an arbitrary record length) 
and reformat the data to write a VB(S) file.  I've known of windows-based 
utilities that process FTPed SMF data (raw or tersed) so the technical 
knowledge is out there.
Date:Wed, 14 Dec 2022 13:22:27 -0600
From:Boesel Guillaume 
Subject: Re: [EXTERNAL] Re: Transmitting SMF records

Hi Rex,
Great. You are right, tersing file from tape to tape works well.
It took around 80-90 MSU during an hour for just one file but it worked.

Hoping that Ituriel will be able to read this file.

Regards and thanks !




Gary Weinhold
Senior Application Architect
DATAKINETICS | Data Performance & Optimization
Phone:+1.613.523.5500 x216
Email: weinh...@dkl.com
Visit us online at 
http://secure-web.cisco.com/1vK3ImFMcgJFz1-podVzEvvVNPbvWjVSieyjkh6kLxBdI18iG5QnqSs8i3kvg0VuH7204uOmMnKxzNWjytZKTENLkVxzqlS6qikvi5yyFIoZCAoeXpVZj5oxOvcUQ3Ser-p1bEnMNTL9HSqHk87DLceE1T02i4JoEtLUqRh-ODRJC2ht43WZl0yLIFPIRwB34PCCKDUw30GrIIxH60SKjQjUQPjlTkmBmKsMJ4ecTK6lrN7uRrBJd5ldQrR3ng7w_T4uwNU_2lim1m5EAEPW1iWz7pc-f38LXw6hw_5oBwp4lS0xjTkyjxSmvz40O8fQljIGe_yml36Zm4s7dRvd4Mtp8Ufji1epM_WSXVoZJ-_wkJRyjjMTjGaoW5QBT-9890J4jC7Ric1OtjYH1N8HRdseY5eoVBhZAd0rmIRykJumh7RV4I3qACvxREuMo4gvP/http%3A%2F%2Fwww.DKL.com
E-mail Notification: The information contained in this email and any 
attachments is confidential and may be subject to copyright or other 
intellectual property protection. If you are not the intended recipient, you 
are not authorized to use or disclose this information, and we request that you 
notify us by reply mail or telephone and delete the original message from your 
mail system.



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Farley, Peter]

Doesn't XMIT/RECEIVE handle VBS in both directions?  I thought it did.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Pommier, Rex
Sent: Thursday, December 15, 2022 10:24 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

What I remember from a discussion long ago was that the reason a VBS dataset 
can't be put back together again is that when the VBS dataset is sent to a 
Windows intermediary, some of the binary metadata defining the records and 
blocks is lost by the Windows side, making it impossible to recreate the VBS on 
the other end.  Whether this is actually true or not, that's what I recall.  
Going straight z/OS to z/OS that metadata is retained.  

Rex

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Gary Weinhold
Sent: Thursday, December 15, 2022 8:44 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Transmitting SMF records

I'm glad there was a solution.

But the underlying problem seems to be that z/OS FTP appears to be limited to 
processing a binary (image) transfer as a stream of bytes unless the transfer 
is between a z/OS client and z/OS server.  IBM RECFM of VB and VBS seem to be 
quite well defined and I don't know why, if the receiving z/OS FTP is informed 
that the data stream is actually VB(S) format, either by SITE/LOCSITE FTP 
commands or by the DCB of a preallocated output dataset, it can't invoke the 
same routines that it would if the sender was a z/OS system.

As an alternative, it wouldn't seem very difficult to create a utility to read 
the FTPed data (received as a sequential file with an arbitrary record length) 
and reformat the data to write a VB(S) file.  I've known of windows-based 
utilities that process FTPed SMF data (raw or tersed) so the technical 
knowledge is out there.
Date:Wed, 14 Dec 2022 13:22:27 -0600
From:Boesel Guillaume 
Subject: Re: [EXTERNAL] Re: Transmitting SMF records

Hi Rex,
Great. You are right, tersing file from tape to tape works well.
It took around 80-90 MSU during an hour for just one file but it worked.

Hoping that Ituriel will be able to read this file.

Regards and thanks !

Gary Weinhold
Senior Application Architect
DATAKINETICS | Data Performance & Optimization
Phone:+1.613.523.5500 x216
Email: weinh...@dkl.com
--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Pommier, Rex]

What I remember from a discussion long ago was that the reason a VBS dataset 
can't be put back together again is that when the VBS dataset is sent to a 
Windows intermediary, some of the binary metadata defining the records and 
blocks is lost by the Windows side, making it impossible to recreate the VBS on 
the other end.  Whether this is actually true or not, that's what I recall.  
Going straight z/OS to z/OS that metadata is retained.  

Rex

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Gary Weinhold
Sent: Thursday, December 15, 2022 8:44 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Transmitting SMF records

I'm glad there was a solution.

But the underlying problem seems to be that z/OS FTP appears to be limited to 
processing a binary (image) transfer as a stream of bytes unless the transfer 
is between a z/OS client and z/OS server.  IBM RECFM of VB and VBS seem to be 
quite well defined and I don't know why, if the receiving z/OS FTP is informed 
that the data stream is actually VB(S) format, either by SITE/LOCSITE FTP 
commands or by the DCB of a preallocated output dataset, it can't invoke the 
same routines that it would if the sender was a z/OS system.

As an alternative, it wouldn't seem very difficult to create a utility to read 
the FTPed data (received as a sequential file with an arbitrary record length) 
and reformat the data to write a VB(S) file.  I've known of windows-based 
utilities that process FTPed SMF data (raw or tersed) so the technical 
knowledge is out there.
Date:Wed, 14 Dec 2022 13:22:27 -0600
From:Boesel Guillaume 
Subject: Re: [EXTERNAL] Re: Transmitting SMF records

Hi Rex,
Great. You are right, tersing file from tape to tape works well.
It took around 80-90 MSU during an hour for just one file but it worked.

Hoping that Ituriel will be able to read this file.

Regards and thanks !




Gary Weinhold
Senior Application Architect
DATAKINETICS | Data Performance & Optimization
Phone:+1.613.523.5500 x216
Email: weinh...@dkl.com
Visit us online at 
https://urldefense.com/v3/__http://www.DKL.com__;!!KjMRP1Ixj6eLE0Fj!t94elsOrEvIcYpn4y0psf9Eq7Ucy2y6VjZr7S8ZoEs891YhFSJHV31zfn4u6nrJDbihyksQwsobJxQ0_zQ$
 
E-mail Notification: The information contained in this email and any 
attachments is confidential and may be subject to copyright or other 
intellectual property protection. If you are not the intended recipient, you 
are not authorized to use or disclose this information, and we request that you 
notify us by reply mail or telephone and delete the original message from your 
mail system.



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
The information contained in this message is confidential, protected from 
disclosure and may be legally privileged. If the reader of this message is not 
the intended recipient or an employee or agent responsible for delivering this 
message to the intended recipient, you are hereby notified that any disclosure, 
distribution, copying, or any action taken or action omitted in reliance on it, 
is strictly prohibited and may be unlawful. If you have received this 
communication in error, please notify us immediately by replying to this 
message and destroy the material in its entirety, whether in electronic or hard 
copy format. Thank you.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Gary Weinhold]

I'm glad there was a solution.

But the underlying problem seems to be that z/OS FTP appears to be limited to 
processing a binary (image) transfer as a stream of bytes unless the transfer 
is between a z/OS client and z/OS server.  IBM RECFM of VB and VBS seem to be 
quite well defined and I don't know why, if the receiving z/OS FTP is informed 
that the data stream is actually VB(S) format, either by SITE/LOCSITE FTP 
commands or by the DCB of a preallocated output dataset, it can't invoke the 
same routines that it would if the sender was a z/OS system.

As an alternative, it wouldn't seem very difficult to create a utility to read 
the FTPed data (received as a sequential file with an arbitrary record length) 
and reformat the data to write a VB(S) file.  I've known of windows-based 
utilities that process FTPed SMF data (raw or tersed) so the technical 
knowledge is out there.
Date:Wed, 14 Dec 2022 13:22:27 -0600
From:Boesel Guillaume 
Subject: Re: [EXTERNAL] Re: Transmitting SMF records

Hi Rex,
Great. You are right, tersing file from tape to tape works well.
It took around 80-90 MSU during an hour for just one file but it worked.

Hoping that Ituriel will be able to read this file.

Regards and thanks !




Gary Weinhold
Senior Application Architect
DATAKINETICS | Data Performance & Optimization
Phone:+1.613.523.5500 x216
Email: weinh...@dkl.com
Visit us online at www.DKL.com
E-mail Notification: The information contained in this email and any 
attachments is confidential and may be subject to copyright or other 
intellectual property protection. If you are not the intended recipient, you 
are not authorized to use or disclose this information, and we request that you 
notify us by reply mail or telephone and delete the original message from your 
mail system.



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[John McKown]

I have not done z/OS to z/OS. But I do download SMF to Windows for SCRT.

I do:

bin
quote site rdw
get zos.smf windows.smf

On Wed, Dec 14, 2022, 07:57 Ituriel do Neto <
03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:

> Hi all,
>
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The
> file got
> corrupted.
>
> I have a customer that has a huge SMF dataset that can't be TERSED or
> XMITTED
> because of a lack of space.
>
> Is there a way to send it, without previous use of XMIT or TRS ?
>
> Thanks in advance.
>
>
> Best Regards
>
> Ituriel do Nascimento Neto
> z/OS System Programmer
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [EXTERNAL] Re: Transmitting SMF records

[Boesel Guillaume]

Hi Rex,
Great. You are right, tersing file from tape to tape works well. 
It took around 80-90 MSU during an hour for just one file but it worked.

Hoping that Ituriel will be able to read this file.

Regards and thanks !


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Pommier, Rex]

Gil,

How does a site not having space to "store tens of gigabytes" of data make IBM 
technically backward?  There are plenty of legitimate concerns about IBM where 
this case can be made but not here.  We're a small mainframe site yet we have 
VSAM datasets ranging upwards of 300 GB.  Large sites have Db2 tables in the 
multiple terabyte range.  My Windows counterparts have disk arrays that 
physically dwarf my DS array.  I would be surprised to find many (if any) 
medium to large sites entrusting their corporate data to consumer grade disks 
that cost fifty to a couple hundred bucks and fit in a shirt pocket.  

Rex

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Paul Gilmartin
Sent: Wednesday, December 14, 2022 12:16 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

On Wed, 14 Dec 2022 16:27:07 +, Keith Gooding wrote:

>... The problem is that as far as I know the receiver code is not 
> available to ISVs. AMAPDUPL processing can be performed by the z/OSMF Problem 
> Management function and it would be nice if ISVs could receive data sent by 
> AMAPDUPL. 
>
RFE.

>You can also direct AMAPDUPL to copy the chunks to a unix file system instead 
>of transmitting them. 
>
>BTW I think z/os ftp may be able to transmit from a unix pipe ...
>
My experience was, it can't.  But that may nave changed.

>... but AMATERSE cannot write to a pipe. 
>
RFE.  Access methods support pipes well.  AMATERSE merely needs to abandon its 
prejudices.

This discussion focuses on lack of space to store tens of gigabytes.  Outside 
the z world that comes for a few hours' pah and fits in a shirt pocket.  Why is 
IBM so technically backward?

--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
The information contained in this message is confidential, protected from 
disclosure and may be legally privileged. If the reader of this message is not 
the intended recipient or an employee or agent responsible for delivering this 
message to the intended recipient, you are hereby notified that any disclosure, 
distribution, copying, or any action taken or action omitted in reliance on it, 
is strictly prohibited and may be unlawful. If you have received this 
communication in error, please notify us immediately by replying to this 
message and destroy the material in its entirety, whether in electronic or hard 
copy format. Thank you.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Farley, Peter]

RE: Mainframe disk space -- Not technically backward, rather financially 
punishing.  Mainframe disk storage costs high-multiple orders of magnitude more 
than server-grade multi-terabyte SSD's.  Yes, better reliability (RAS in 
general) should cost more, but that much more?

Just another case of what one of the IBM CEO's promised at an investor's 
meeting decades ago - IBM will never stay in any low-margin business.  They 
have largely kept that promise.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Paul Gilmartin
Sent: Wednesday, December 14, 2022 12:16 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Transmitting SMF records

On Wed, 14 Dec 2022 16:27:07 +, Keith Gooding wrote:

>... The problem is that as far as I know the receiver code is not 
> available to ISVs. AMAPDUPL processing can be performed by the z/OSMF Problem 
> Management function and it would be nice if ISVs could receive data sent by 
> AMAPDUPL. 
>
RFE.

>You can also direct AMAPDUPL to copy the chunks to a unix file system instead 
>of transmitting them. 
>
>BTW I think z/os ftp may be able to transmit from a unix pipe ...
>
My experience was, it can't.  But that may nave changed.

>... but AMATERSE cannot write to a pipe. 
>
RFE.  Access methods support pipes well.  AMATERSE merely needs to abandon its 
prejudices.

This discussion focuses on lack of space to store tens of gigabytes.  Outside 
the z world that comes for a few hours' pah and fits in a shirt pocket.  Why is 
IBM so technically backward?

--

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Wed, 14 Dec 2022 16:27:07 +, Keith Gooding wrote:

>... The problem is that as far as I know the receiver code is not 
> available to ISVs. AMAPDUPL processing can be performed by the z/OSMF Problem 
> Management function and it would be nice if ISVs could receive data sent by 
> AMAPDUPL. 
>
RFE.

>You can also direct AMAPDUPL to copy the chunks to a unix file system instead 
>of transmitting them. 
>
>BTW I think z/os ftp may be able to transmit from a unix pipe ...
>
My experience was, it can't.  But that may nave changed.

>... but AMATERSE cannot write to a pipe. 
>
RFE.  Access methods support pipes well.  AMATERSE merely needs to abandon
its prejudices.

This discussion focuses on lack of space to store tens of gigabytes.  Outside 
the
z world that comes for a few hours' pah and fits in a shirt pocket.  Why is IBM
so technically backward?

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [EXTERNAL] Re: Transmitting SMF records

[Pommier, Rex]

Hi Guillaume,

I take it these SMF files are on tape.  Can terse take a tape dataset as input 
and write out a tersed tape?  If so, and you're then PUTting the file to 
Ituriel, FTP should work fine reading the file from tape to transmit it to the 
intermediary Windows box.  We do this on a regular basis, FTPing large (~100 
GB) files straight from tape to Windows.

Rex

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Boesel Guillaume
Sent: Wednesday, December 14, 2022 10:18 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [EXTERNAL] Re: Transmitting SMF records

Hi all,
"I have a customer that has a huge SMF dataset"

I'm the "customer" who sent these huge SMF datasets.
We are not able to offload them on DASD to terse or xmit them, there are really 
to much huge (around 80-100 GB per files), we don't have enough disk space.

I sent the same files to IBM using their AMAPDUPL tool without problem.
IBM was able to read them. 

But with Ituriel, I can't use AMAPDUPL and I have to use FTP.

i used these job to transfer the SMF tape file to a intern windows FTP server. 
After, I used filezilla to transfer from intern windows server to Ituriel 
server.

//STEP1 EXEC PGM=FTP   
//FTPDATA DD DISP=SHR,DSN=USERID.FTPDATA.FWFRIEND  
//* FTPDATA.FWFRIEND CONTAINS "FWFRIENDLY TRUE"
//FTPFIL1 DD DISP=SHR,DSN=SMF.REC110.SAVJOUR.G6034V00,   
//   DCB=(RECFM=U)  
  
//OUTPUT DD SYSOUT=*
//INPUT  DD * 
server_name 21 
user password
binary
cd / 
pwd   
put //DD:FTPFIL1  XXX.CICS.SMF.XXX.J297.A2022   
close

As a test, I used AMAPDUPL to send SMF file to Ituriel, it transferred around 
ten files. It could be a solution if we know how to reassemble AMAPDUPL files...

Thanks
Guillaume

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
The information contained in this message is confidential, protected from 
disclosure and may be legally privileged. If the reader of this message is not 
the intended recipient or an employee or agent responsible for delivering this 
message to the intended recipient, you are hereby notified that any disclosure, 
distribution, copying, or any action taken or action omitted in reliance on it, 
is strictly prohibited and may be unlawful. If you have received this 
communication in error, please notify us immediately by replying to this 
message and destroy the material in its entirety, whether in electronic or hard 
copy format. Thank you.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Paul Gilmartin]

On Wed, 14 Dec 2022 13:56:59 +, Ituriel do Neto  wrote:
>
>I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
>because of a lack of space.
>
>Is there a way to send it, without previous use of XMIT or TRS ?
>
This feels like a motivation for an RFE for AMATERSE to support UNIX files as 
its
packed archives.  Thise UNIX files might then be POSIX pipes or network sockets.
eliminating any need for huge temporary data sets.

My experience has been that AMATERSE will not directly unpack a UNIX file.
However, if SYSUT1 is a concatenation of an empty PS data set and that same
UNIX file, it unpacks successfully.  This feels not like a technically valid 
reason
but simply anti-UNIX bigotry.

Alas, there's no such workaround for PACK SYSUT2.

Would AWSTAPE be a transmittable format?

But, beware, if the data are too big to fit on disk, they're at risk of network
reliability and bandwidth limitations.

Why do you feel Windows is needed?  It adds needless complexity.

-- 
gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Keith Gooding]

Although not a solution to your problem you may know that the z/os AMAPDUPL 
utility solves this problem by automatically tersing the data, splitting it 
into chunks, and transmitting the chunks to the IBM support site in a number of 
overlapping ftp or https  streams. I think it  uses pipes to overlap some of 
the terse processing with transmission. At the other end a receiving process 
automatically re-assembles. The problem is that as far as I know the receiver 
code is not available to ISVs. AMAPDUPL processing can be performed by the 
z/OSMF Problem Management function and it would be nice if ISVs could receive 
data sent by AMAPDUPL. 

You can also direct AMAPDUPL to copy the chunks to a unix file system instead 
of transmitting them. 

BTW I think z/os ftp may be able to transmit from a unix pipe but AMATERSE 
cannot write to a pipe. 


  
Keith

> On 14 Dec 2022, at 13:57, Ituriel do Neto 
> <03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:
> 
> Hi all,
> 
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form 
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered 
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then 
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The file 
> got 
> corrupted.
> 
> I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
> because of a lack of space.
> 
> Is there a way to send it, without previous use of XMIT or TRS ?
> 
> Thanks in advance.
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Boesel Guillaume]

Hi all,
"I have a customer that has a huge SMF dataset"

I'm the "customer" who sent these huge SMF datasets.
We are not able to offload them on DASD to terse or xmit them, there are really 
to much huge (around 80-100 GB per files), we don't have enough disk space.

I sent the same files to IBM using their AMAPDUPL tool without problem.
IBM was able to read them. 

But with Ituriel, I can't use AMAPDUPL and I have to use FTP.

i used these job to transfer the SMF tape file to a intern windows FTP server. 
After, I used filezilla to transfer from intern windows server to Ituriel 
server.

//STEP1 EXEC PGM=FTP   
//FTPDATA DD DISP=SHR,DSN=USERID.FTPDATA.FWFRIEND  
//* FTPDATA.FWFRIEND CONTAINS "FWFRIENDLY TRUE"
//FTPFIL1 DD DISP=SHR,DSN=SMF.REC110.SAVJOUR.G6034V00,   
//   DCB=(RECFM=U)  
  
//OUTPUT DD SYSOUT=*
//INPUT  DD * 
server_name 21 
user password
binary
cd / 
pwd   
put //DD:FTPFIL1  XXX.CICS.SMF.XXX.J297.A2022   
close

As a test, I used AMAPDUPL to send SMF file to Ituriel, it transferred around 
ten files. It could be a solution if we know how to reassemble AMAPDUPL files...

Thanks
Guillaume

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Radoslaw Skorupka]

My €0.02:
1. Be specific. Provide us as many details as needed.  Huge size, lack 
of space for TERSE (really)? - all those details were provided later.

2. CHANGE SOMETHING. You have to change something.
3. First I would discuss the size of SMF dataset. SMF repository need 
not to be single huge dataset. It can be series of daily datasets as well.
4. There are many ways to transmit dataset z/OS to z/OS. Some of them 
are commercial products. Do you want to pay for Managed File Transfer 
product and avoid boring XMIT/TERSE/zip/NJE/whatever methods?
5. Avoid Windows between source and target z/OS. Otherwise use XMIT, 
TERSE or any other method for packing datasets.
6. Consider shared DASD or shared tape instead of LAN. Yes, you did not 
tell us about whether both systems can share DASD/tape. ;-)



HTH

--
Radoslaw Skorupka
Lodz, Poland



W dniu 14.12.2022 o 14:56, Ituriel do Neto pisze:

Hi all,

I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form dataset,
that can be downloaded in binary mode, transmitted, and then recovered following
the reverse order.
My attempts of downloading the SMF dataset directly, in binary, and then 
uploading
it to another SMF dataset with the same DCB attributes did not work. The file 
got
corrupted.

I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
because of a lack of space.

Is there a way to send it, without previous use of XMIT or TRS ?

Thanks in advance.


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

[Paul Gorlinsky]

If tape is shared... Then use IEBGENER ( ICEGENER ) to copy the XMIT dataset to 
tape... and restore that DSN from tape to feed RECEIVE

FTP is horrible to use ... too much latencies ... Using TERSE in there adds 
verification that the data is intact ...

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

[Farley, Peter]

But does your FTP server on z/OS allow you to use GET from a tape device?  Tape 
allocations are sometimes not permitted from an active server address space so 
as not to hold up production work using the same server.

Just a possible roadblock, not a given, depends on your setup.

Peter

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
rpinion865
Sent: Wednesday, December 14, 2022 10:10 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

Like I said earlier, you can TERSE to a tape dataset.  I use that method all of 
the time, for transferring mainframe data between mainframe systems, that have 
an intervening non-mainframe repository.

Sent with Proton Mail secure email.

--- Original Message ---
On Wednesday, December 14th, 2022 at 10:06 AM, Lionel B. Dyck 
 wrote:


> Be aware that your installation may have a limit on the size of a Transmit 
> (XMIT) file. Issue TSO PARMLIB and look for the OUTLIM value - if too small 
> you may want to increase it - the same for OUTWARN which reports every 
> increment of the XMIT file.
> 
> Using TERSE is probably the better solution as you won't have those limits 
> and works just as well for your purpose.
> 
> Lionel B. Dyck <><
> 
> Website: 
> https://urldefense.com/v3/__https://www.lbdsoftware.com__;!!Ebr-cpPeAn
> fNniQ8HSAI-g_K5b7VKg!JoX_FuFXWLHmUkgpXig9Vag_99TNbdAMo7FvmRE3WWT2UqLNQ
> bln2Lv0qs8A8xS4bZtC0p8gJykKcwsdu9EwKUP_4eZAIKwpb8K_m3HX$
> Github: 
> https://urldefense.com/v3/__https://github.com/lbdyck__;!!Ebr-cpPeAnfN
> niQ8HSAI-g_K5b7VKg!JoX_FuFXWLHmUkgpXig9Vag_99TNbdAMo7FvmRE3WWT2UqLNQbl
> n2Lv0qs8A8xS4bZtC0p8gJykKcwsdu9EwKUP_4eZAIKwpb3Vtui78$
> 
> “Worry more about your character than your reputation. Character is 
> what you are, reputation merely what others think you are.” - - - John 
> Wooden
> 
> -Original Message-
> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On Behalf Of 
> Crawford, Robert C.
> 
> Sent: Wednesday, December 14, 2022 8:55 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: [Public] RE: EXTERNAL: Re: Transmitting SMF records
> 
> Sorry, I see what you mean.
> 
> I've installed a couple of IBM tools from my Windows workstation that were in 
> XMIT format. I uploaded them to z/OS with binary FTP then did a RECEIVE to 
> recreate the file. I guess you could do the same by XMIT'ing to a dataset, 
> downloading that to Windows, to Windows, to z/OS where you do a RECEIVE. All 
> in binary format, of course.
> 
> Robert Crawford
> Mainframe Management
> United Services Automobile Association
> (210) 913-3822
> 
> “Nothing can be beautiful which is not true."
> John Ruskin
> Please send requests to mainframe management through our front door at 
> go/mfmfrontdoor
> 
> -Original Message-
> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On Behalf 
> Of Ituriel do Neto
> 
> Sent: Wednesday, December 14, 2022 8:36 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: EXTERNAL: Re: Transmitting SMF records
> 
> The idea is to send an SMF dataset from one z/OS to another one, but first, 
> it needs to be downloaded to windows to be sent to us, also in windows. Once 
> we have the file, we upload it to our mainframe to process it.
> 
> It is possible to split the original SMF dataset in smaller pieces but 
> demands a lot of control, so it would be the last resource..
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> 
> 
> 
> 
> 
> Em quarta-feira, 14 de dezembro de 2022 11:06:35 BRT, Matt Hogstrom 
> m...@hogstrom.org escreveu:
> 
> 
> 
> 
> 
> 
> Are you processing this on another z/OS system ? You indicated you’re 
> downloading it so I wanted to make sure I understand the requirements.
> 
> For “downloading” are you using FTP, SFTP or SCP. Are you processing the data 
> on a non-Z platform ?
> 
> Matt Hogstrom
> m...@hogstrom.org
> 
> “It may be cognitive, but, it ain’t intuitive."
> — Hogstrom
> 
> 
> > On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> > 03427ec2837d-dmarc-requ...@listserv.ua.edu wrote:
> > 
> > Hi all,
> > 
> > I know we can TERSE or use XMIT a SMF dataset to generate a 
> > fixed-form dataset, that can be downloaded in binary mode, 
> > transmitted, and then recovered following the reverse order.
> > My attempts of downloading the SMF dataset directly, in binary, and 
> > then uploading it to another SMF dataset with the same DCB 
> > attributes did not work. The file got corrupted.
> > 
> > I have a customer that has a huge SMF dataset that can't be TERSED 
> > or XMITTED because of a lac

Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

[Paul Gorlinsky]

SMF data is usually VBS format, Variable Block Span. Physical blocks 4K but 
records can be much larger.

First question: Are the two z/OS machines connected via TCP/IP ? if so TERSE 
XMIT to the 2nd machine, RECEIVE, DETERSE else 
Second: Is the data VB or still VBS?

Using TERSE and then XMIT to a dataset, does two things, It compresses the data 
into 1K records and then to 80 byte records of FIXED length. 

The FIXED LENGTH simplifies your life... 

This makes downloading to the PC and uploading to the 2nd zOS almost bullet 
proof.

Download as Binary file and upload as a Binary file of RECFM FB, LRECL 80, 
BLKSIZE 0.

Then RECEIVE the file and DETERSE it. 

This should be an exact copy at that point.

The other option is to use ADRDSSU and dump to a dataset the SMF data, the 
TERSE and XMIT --- RECEIVE, DETERSE, RESTORE

Paul

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

[rpinion865]

Like I said earlier, you can TERSE to a tape dataset.  I use that method all of 
the time, for transferring mainframe data between mainframe systems, that have 
an intervening non-mainframe repository.




Sent with Proton Mail secure email.

--- Original Message ---
On Wednesday, December 14th, 2022 at 10:06 AM, Lionel B. Dyck 
 wrote:


> Be aware that your installation may have a limit on the size of a Transmit 
> (XMIT) file. Issue TSO PARMLIB and look for the OUTLIM value - if too small 
> you may want to increase it - the same for OUTWARN which reports every 
> increment of the XMIT file.
> 
> Using TERSE is probably the better solution as you won't have those limits 
> and works just as well for your purpose.
> 
> Lionel B. Dyck <><
> 
> Website: https://www.lbdsoftware.com
> Github: https://github.com/lbdyck
> 
> “Worry more about your character than your reputation. Character is what you 
> are, reputation merely what others think you are.” - - - John Wooden
> 
> -Original Message-
> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On Behalf Of 
> Crawford, Robert C.
> 
> Sent: Wednesday, December 14, 2022 8:55 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: [Public] RE: EXTERNAL: Re: Transmitting SMF records
> 
> Sorry, I see what you mean.
> 
> I've installed a couple of IBM tools from my Windows workstation that were in 
> XMIT format. I uploaded them to z/OS with binary FTP then did a RECEIVE to 
> recreate the file. I guess you could do the same by XMIT'ing to a dataset, 
> downloading that to Windows, to Windows, to z/OS where you do a RECEIVE. All 
> in binary format, of course.
> 
> Robert Crawford
> Mainframe Management
> United Services Automobile Association
> (210) 913-3822
> 
> “Nothing can be beautiful which is not true."
> John Ruskin
> Please send requests to mainframe management through our front door at 
> go/mfmfrontdoor
> 
> -Original Message-
> From: IBM Mainframe Discussion List IBM-MAIN@LISTSERV.UA.EDU On Behalf Of 
> Ituriel do Neto
> 
> Sent: Wednesday, December 14, 2022 8:36 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: EXTERNAL: Re: Transmitting SMF records
> 
> The idea is to send an SMF dataset from one z/OS to another one, but first, 
> it needs to be downloaded to windows to be sent to us, also in windows. Once 
> we have the file, we upload it to our mainframe to process it.
> 
> It is possible to split the original SMF dataset in smaller pieces but 
> demands a lot of control, so it would be the last resource..
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> 
> 
> 
> 
> 
> Em quarta-feira, 14 de dezembro de 2022 11:06:35 BRT, Matt Hogstrom 
> m...@hogstrom.org escreveu:
> 
> 
> 
> 
> 
> 
> Are you processing this on another z/OS system ? You indicated you’re 
> downloading it so I wanted to make sure I understand the requirements.
> 
> For “downloading” are you using FTP, SFTP or SCP. Are you processing the data 
> on a non-Z platform ?
> 
> Matt Hogstrom
> m...@hogstrom.org
> 
> “It may be cognitive, but, it ain’t intuitive."
> — Hogstrom
> 
> 
> > On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> > 03427ec2837d-dmarc-requ...@listserv.ua.edu wrote:
> > 
> > Hi all,
> > 
> > I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form
> > dataset, that can be downloaded in binary mode, transmitted, and then
> > recovered following the reverse order.
> > My attempts of downloading the SMF dataset directly, in binary, and
> > then uploading it to another SMF dataset with the same DCB attributes
> > did not work. The file got corrupted.
> > 
> > I have a customer that has a huge SMF dataset that can't be TERSED or
> > XMITTED because of a lack of space.
> > 
> > Is there a way to send it, without previous use of XMIT or TRS ?
> > 
> > Thanks in advance.
> > 
> > Best Regards
> > 
> > Ituriel do Nascimento Neto
> > z/OS System Programmer
> > 
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions, send
> > email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
> lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send ema

Re: [Public] RE: EXTERNAL: Re: Transmitting SMF records

[Lionel B. Dyck]

Be aware that your installation may have a limit on the size of a Transmit 
(XMIT) file.  Issue TSO PARMLIB and look for the OUTLIM value - if too small 
you may want to increase it - the same for OUTWARN which reports every 
increment of the XMIT file.

Using TERSE is probably the better solution as you won't have those limits and 
works just as well for your purpose.

Lionel B. Dyck <><
Website: https://www.lbdsoftware.com
Github: https://github.com/lbdyck

“Worry more about your character than your reputation. Character is what you 
are, reputation merely what others think you are.”   - - - John Wooden

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Crawford, Robert C.
Sent: Wednesday, December 14, 2022 8:55 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: [Public] RE: EXTERNAL: Re: Transmitting SMF records

Sorry, I see what you mean.

I've installed a couple of IBM tools from my Windows workstation that were in 
XMIT format.  I uploaded them to z/OS with binary FTP then did a RECEIVE to 
recreate the file.  I guess you could do the same by XMIT'ing to a dataset, 
downloading that to Windows, to Windows, to z/OS where you do a RECEIVE.  All 
in binary format, of course.

Robert Crawford
Mainframe Management
United Services Automobile Association
(210) 913-3822

“Nothing can be beautiful which is not true."
John Ruskin
Please send requests to mainframe management through our front door at  
go/mfmfrontdoor

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Ituriel do Neto
Sent: Wednesday, December 14, 2022 8:36 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: EXTERNAL: Re: Transmitting SMF records

The idea is to send an SMF dataset from one z/OS to another one, but first, it 
needs to be downloaded to windows to be sent to us, also in windows. Once we 
have the file, we upload it to our mainframe to process it.

It is possible to split the original SMF dataset in smaller pieces but demands 
a lot of control, so it would be the last resource..


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer






Em quarta-feira, 14 de dezembro de 2022 11:06:35 BRT, Matt Hogstrom 
 escreveu:





Are you processing this on another z/OS system ?  You indicated you’re 
downloading it so I wanted to make sure I understand the requirements.

For “downloading”  are you using FTP, SFTP or SCP.  Are you processing the data 
on a non-Z platform ?

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom



> On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> <03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:
>
> Hi all,
>
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form 
> dataset, that can be downloaded in binary mode, transmitted, and then 
> recovered following the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and 
> then uploading it to another SMF dataset with the same DCB attributes 
> did not work. The file got corrupted.
>
> I have a customer that has a huge SMF dataset that can't be TERSED or 
> XMITTED because of a lack of space.
>
> Is there a way to send it, without previous use of XMIT or TRS ?
>
> Thanks in advance.
>
>
> Best Regards
>
> Ituriel do Nascimento Neto
> z/OS System Programmer
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

USAA Classification: Public

Disclaimer: This email and any attachments are the property of USAA and may 
contain confidential and/or privileged material. If you are not the intended 
recipient, any use, disclosure or copying of this email or any attachments is 
unauthorized. If you received this email in error, please immediately notify 
the sender and delete the email and any attachments from your computer.


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

[Public] RE: EXTERNAL: Re: Transmitting SMF records

[Crawford, Robert C.]

Sorry, I see what you mean.

I've installed a couple of IBM tools from my Windows workstation that were in 
XMIT format.  I uploaded them to z/OS with binary FTP then did a RECEIVE to 
recreate the file.  I guess you could do the same by XMIT'ing to a dataset, 
downloading that to Windows, to Windows, to z/OS where you do a RECEIVE.  All 
in binary format, of course.

Robert Crawford
Mainframe Management
United Services Automobile Association
(210) 913-3822

“Nothing can be beautiful which is not true."
John Ruskin
Please send requests to mainframe management through our front door at  
go/mfmfrontdoor

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Ituriel do Neto
Sent: Wednesday, December 14, 2022 8:36 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: EXTERNAL: Re: Transmitting SMF records

The idea is to send an SMF dataset from one z/OS to another one, but first, it 
needs to be downloaded to windows to be sent to us, also in windows. Once we 
have the file, we upload it to our mainframe to process it.

It is possible to split the original SMF dataset in smaller pieces but demands 
a lot of control, so it would be the last resource..


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer






Em quarta-feira, 14 de dezembro de 2022 11:06:35 BRT, Matt Hogstrom 
 escreveu:





Are you processing this on another z/OS system ?  You indicated you’re 
downloading it so I wanted to make sure I understand the requirements.

For “downloading”  are you using FTP, SFTP or SCP.  Are you processing the data 
on a non-Z platform ?

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom



> On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> <03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:
>
> Hi all,
>
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form
> dataset, that can be downloaded in binary mode, transmitted, and then
> recovered following the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and
> then uploading it to another SMF dataset with the same DCB attributes
> did not work. The file got corrupted.
>
> I have a customer that has a huge SMF dataset that can't be TERSED or
> XMITTED because of a lack of space.
>
> Is there a way to send it, without previous use of XMIT or TRS ?
>
> Thanks in advance.
>
>
> Best Regards
>
> Ituriel do Nascimento Neto
> z/OS System Programmer
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

USAA Classification: Public

Disclaimer: This email and any attachments are the property of USAA and may 
contain confidential and/or privileged material. If you are not the intended 
recipient, any use, disclosure or copying of this email or any attachments is 
unauthorized. If you received this email in error, please immediately notify 
the sender and delete the email and any attachments from your computer.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Ituriel do Neto]

The idea is to send an SMF dataset from one z/OS to another one, but first, it 
needs to be downloaded to windows to be sent to us, also in windows. Once we 
have the file, we upload it to our mainframe to process it.

It is possible to split the original SMF dataset in smaller pieces but demands 
a lot of control, so it would be the last resource..


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer






Em quarta-feira, 14 de dezembro de 2022 11:06:35 BRT, Matt Hogstrom 
 escreveu: 





Are you processing this on another z/OS system ?  You indicated you’re 
downloading it so I wanted to make sure I understand the requirements.

For “downloading”  are you using FTP, SFTP or SCP.  Are you processing the data 
on a non-Z platform ?

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom



> On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> <03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:
> 
> Hi all,
> 
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form 
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered 
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then 
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The file 
> got 
> corrupted.
> 
> I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
> because of a lack of space.
> 
> Is there a way to send it, without previous use of XMIT or TRS ?
> 
> Thanks in advance.
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

[Public] RE: EXTERNAL: Transmitting SMF records

[Crawford, Robert C.]

I've used XMIT to send small SMF files between Sysplex's.  The catch is the 
transmitted file occupies spool space until someone receives it.  If your shop 
is hurting for space they probably don't have a lot of spool volumes either.

Robert Crawford
Mainframe Management
United Services Automobile Association
(210) 913-3822

“Nothing can be beautiful which is not true."
John Ruskin
Please send requests to mainframe management through our front door at  
go/mfmfrontdoor

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Ituriel do Neto
Sent: Wednesday, December 14, 2022 7:57 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: EXTERNAL: Transmitting SMF records

Hi all,

I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form dataset, 
that can be downloaded in binary mode, transmitted, and then recovered 
following the reverse order.
My attempts of downloading the SMF dataset directly, in binary, and then 
uploading it to another SMF dataset with the same DCB attributes did not work. 
The file got corrupted.

I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED 
because of a lack of space.

Is there a way to send it, without previous use of XMIT or TRS ?

Thanks in advance.


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

USAA Classification: Public

Disclaimer: This email and any attachments are the property of USAA and may 
contain confidential and/or privileged material. If you are not the intended 
recipient, any use, disclosure or copying of this email or any attachments is 
unauthorized. If you received this email in error, please immediately notify 
the sender and delete the email and any attachments from your computer.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Matt Hogstrom]

Are you processing this on another z/OS system ?   You indicated you’re 
downloading it so I wanted to make sure I understand the requirements.

For “downloading”  are you using FTP, SFTP or SCP.  Are you processing the data 
on a non-Z platform ?

Matt Hogstrom
m...@hogstrom.org

“It may be cognitive, but, it ain’t intuitive."
— Hogstrom



> On Dec 14, 2022, at 8:56 AM, Ituriel do Neto 
> <03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:
> 
> Hi all,
> 
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form 
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered 
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then 
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The file 
> got 
> corrupted.
> 
> I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
> because of a lack of space.
> 
> Is there a way to send it, without previous use of XMIT or TRS ?
> 
> Thanks in advance.
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Itschak Mugzach]

Why don't you divide the huge file into sections? TRY idcams copy
fromnumber with count

*| **Itschak Mugzach | Director | SecuriTeam Software **|** IronSphere
Platform* *|* *Information Security Continuous Monitoring for Z/OS, zLinux
and IBM I **|  *

*|* *Email**: i_mugz...@securiteam.co.il **|* *Mob**: +972 522 986404 **|*
*Skype**: ItschakMugzach **|* *Web**: www.Securiteam.co.il  **|*





On Wed, Dec 14, 2022 at 3:57 PM Ituriel do Neto <
03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:

> Hi all,
>
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The
> file got
> corrupted.
>
> I have a customer that has a huge SMF dataset that can't be TERSED or
> XMITTED
> because of a lack of space.
>
> Is there a way to send it, without previous use of XMIT or TRS ?
>
> Thanks in advance.
>
>
> Best Regards
>
> Ituriel do Nascimento Neto
> z/OS System Programmer
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[Colin Paice]

You could try extracting a subset of records based on start time and end
time... then xmiting those, and at the far end concatenation them
Colin

On Wed, 14 Dec 2022 at 13:57, Ituriel do Neto <
03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:

> Hi all,
>
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The
> file got
> corrupted.
>
> I have a customer that has a huge SMF dataset that can't be TERSED or
> XMITTED
> because of a lack of space.
>
> Is there a way to send it, without previous use of XMIT or TRS ?
>
> Thanks in advance.
>
>
> Best Regards
>
> Ituriel do Nascimento Neto
> z/OS System Programmer
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Transmitting SMF records

[rpinion865]

You can TERSE to a "tape" dataset.




Sent with Proton Mail secure email.

--- Original Message ---
On Wednesday, December 14th, 2022 at 8:56 AM, Ituriel do Neto 
<03427ec2837d-dmarc-requ...@listserv.ua.edu> wrote:


> Hi all,
> 
> I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form 
> dataset,
> that can be downloaded in binary mode, transmitted, and then recovered 
> following
> the reverse order.
> My attempts of downloading the SMF dataset directly, in binary, and then 
> uploading
> it to another SMF dataset with the same DCB attributes did not work. The file 
> got
> corrupted.
> 
> I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
> because of a lack of space.
> 
> Is there a way to send it, without previous use of XMIT or TRS ?
> 
> Thanks in advance.
> 
> 
> Best Regards
> 
> Ituriel do Nascimento Neto
> z/OS System Programmer
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Transmitting SMF records

[Ituriel do Neto]

Hi all,

I know we can TERSE or use XMIT a SMF dataset to generate a fixed-form dataset,
that can be downloaded in binary mode, transmitted, and then recovered following
the reverse order.
My attempts of downloading the SMF dataset directly, in binary, and then 
uploading
it to another SMF dataset with the same DCB attributes did not work. The file 
got 
corrupted.

I have a customer that has a huge SMF dataset that can't be TERSED or XMITTED
because of a lack of space.

Is there a way to send it, without previous use of XMIT or TRS ?

Thanks in advance.


Best Regards

Ituriel do Nascimento Neto
z/OS System Programmer

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Format SMF records created by SYNCSORT MFX

[Gadi Ben-Avi]

Hi Max,
I found that MXG can format the Syncsort SMF recrods, so we'll be using that.
Thanks for your help

Gadi

-Original Message-
From: IBM Mainframe Discussion List  On Behalf Of 
Massimo Biancucci
Sent: Wednesday, July 27, 2022 11:33 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Format SMF records created by SYNCSORT MFX

Gadi,

I wrote a REXX that formats most of the relevant fields.

I'm not sure it's a so "common need", so contact me offline and I'll be happy 
to help.

Best regards.
Max

Il giorno mar 26 lug 2022 alle ore 13:16 Gadi Ben-Avi  ha
scritto:

> Hi,
> Does anyone know of a, hopefully free,  tool that can format Syncsort 
> MXF
> v3.1 SMF records?
>
> Thanks
>
> Gadi
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send 
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Email secured by Check Point

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Format SMF records created by SYNCSORT MFX

[Massimo Biancucci]

Gadi,

I wrote a REXX that formats most of the relevant fields.

I'm not sure it's a so "common need", so contact me offline and I'll be
happy to help.

Best regards.
Max

Il giorno mar 26 lug 2022 alle ore 13:16 Gadi Ben-Avi  ha
scritto:

> Hi,
> Does anyone know of a, hopefully free,  tool that can format Syncsort MXF
> v3.1 SMF records?
>
> Thanks
>
> Gadi
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Format SMF records created by SYNCSORT MFX

[Gadi Ben-Avi]

Hi,
Does anyone know of a, hopefully free,  tool that can format Syncsort MXF v3.1 
SMF records?

Thanks

Gadi

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF records

[kekronbekron]

Hi Peter,

a) enabling as many subtypes in 119
b) check out near real-time monitoring for z/OS CS - 
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.4.0/com.ibm.zos.v2r4.halx001/inttcp.htm
c) doing whatever's required (sorry I don't know the specifics off the top of 
my head) to make RACF write as much as required/possible to type 80

So basically, network & RACF related.

- KB

‐‐‐ Original Message ‐‐‐
On Friday, July 17, 2020 8:53 PM, TenEyck, Peter 
 wrote:

> A general question; from a security and auditing perspective, is there a best 
> practices or recommendation of what SMF records and sub types should be 
> created?
>
> //* Peter Ten Eyck
> //* Senior Systems Programmer
> //* American National
> //
>
> American National is the brand name for American National Insurance Company, 
> headquartered in Galveston, Texas, and its subsidiaries. Each company has 
> financial responsibility only for its own products and services. American 
> National Insurance Company is not licensed in New York. In New York, business 
> is conducted by New York licensed subsidiaries. For more information, go to 
> www.americannational.com.
> Confidentiality: This transmission, including any attachments, is solely for 
> the use of the intended recipient(s). This transmission may contain 
> information that is confidential or otherwise protected from disclosure. The 
> use or disclosure of the information contained in this transmission, 
> including any attachments, for any purpose other than that intended by its 
> transmittal is strictly prohibited. Unauthorized interception of this email 
> is a violation of federal criminal law. If you are not an intended recipient 
> of this transmission, please immediately destroy all copies received and 
> notify the sender.
>
> 
>
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

SMF records

[TenEyck, Peter]

A general question; from a security and auditing perspective, is there a best 
practices or recommendation of what SMF records and sub types should be created?

//* Peter Ten Eyck
//* Senior Systems Programmer
//* American National
//



American National is the brand name for American National Insurance Company, 
headquartered in Galveston, Texas, and its subsidiaries. Each company has 
financial responsibility only for its own products and services. American 
National Insurance Company is not licensed in New York. In New York, business 
is conducted by New York licensed subsidiaries. For more information, go to 
www.americannational.com.
Confidentiality: This transmission, including any attachments, is solely for 
the use of the intended recipient(s). This transmission may contain information 
that is confidential or otherwise protected from disclosure. The use or 
disclosure of the information contained in this transmission, including any 
attachments, for any purpose other than that intended by its transmittal is 
strictly prohibited. Unauthorized interception of this email is a violation of 
federal criminal law. If you are not an intended recipient of this 
transmission, please immediately destroy all copies received and notify the 
sender.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Bernd Oppolzer]

This does not give you call relations, where programs are statically 
linked.


And static analysis (related on source code) does not give you relations,
where the name of the callee comes from a table or is computed at run time.

That's what a former customer did (besides static analysis, which was 
done, too):


every program had a startup macro (no matter, if PL/1, ASSEMBLER, or C).
In ASSEMBLER, the startup macro did all the housekeeping, too, of course.

This startup macro, on first call, wrote a WTO message, like:
"I am module  EP  called by  compiled at  "
only once.

At the end of the day, all production jobs and dialog regions (IMS) etc. 
were

scanned for these message types and the call relations were written to
a repository :-)

and this was done day by day.

Kind regards

Bernd



Am 13.11.2019 um 22:07 schrieb Mike Schwab:

http://www.longpelaexpertise.com.au/ezine/SoftwareUsageWithoutTADz.php

For many years IBMs Peter Relson has offered a free ‘as is, no
warranty’ tool to monitor program fetch activity: the Module Fetch
Monitor. Contact Peter directly at rel...@us.ibm.com.




--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Beverly Caldwell]

Write your own CSV exit to knock out an SMF record for these called
programs. I have done this several times and only once missed anything
because some smartie pants from CA was doing his own program loads.

On Thu, Nov 14, 2019 at 3:59 AM Massimo Biancucci  wrote:

> Hi,
>
> as other colleagues already said, no standard SMF way.
>
> At my customers sites, they use TADz (it seems Ptracker gives more
> information like the chain between caller and called) and for some of them
> a product we developed to correlate CICS-Transactions with LINKed/CALLed
> programs (written to SMF records).
>
> About cleanup, if applicable, keep attention to static linked programs who
> seem to be called by nobody.
>
> Best regards.
> Max
>
> Il giorno mer 13 nov 2019 alle ore 17:07 Steff Gladstone <
> steff.gladst...@gmail.com> ha scritto:
>
> > We would like to clean up our load libraries by deleting unused programs.
> >
> > Is there any way to use SMF data to track real-time usage of programs
> which
> > are not main programs but are called by other programs?
> >
> > We were under the impression  that only executions of main programs
> (PGM=)
> >  were recorded with SMF.  Scanning program sources for called programs is
> > unsatisfactory for us since the call could be dynamic (name of program
> > contained in a variable) or conditional on a particular set of
> > circumstances that never occurs in practice.
> >
> > Thanks in advance!
> >
> > Steff Gladstone
> >
> > --
> > For IBM-MAIN subscribe / signoff / archive access instructions,
> > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> >
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Massimo Biancucci]

Hi,

as other colleagues already said, no standard SMF way.

At my customers sites, they use TADz (it seems Ptracker gives more
information like the chain between caller and called) and for some of them
a product we developed to correlate CICS-Transactions with LINKed/CALLed
programs (written to SMF records).

About cleanup, if applicable, keep attention to static linked programs who
seem to be called by nobody.

Best regards.
Max

Il giorno mer 13 nov 2019 alle ore 17:07 Steff Gladstone <
steff.gladst...@gmail.com> ha scritto:

> We would like to clean up our load libraries by deleting unused programs.
>
> Is there any way to use SMF data to track real-time usage of programs which
> are not main programs but are called by other programs?
>
> We were under the impression  that only executions of main programs (PGM=)
>  were recorded with SMF.  Scanning program sources for called programs is
> unsatisfactory for us since the call could be dynamic (name of program
> contained in a variable) or conditional on a particular set of
> circumstances that never occurs in practice.
>
> Thanks in advance!
>
> Steff Gladstone
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Mike Schwab]

http://www.longpelaexpertise.com.au/ezine/SoftwareUsageWithoutTADz.php

For many years IBMs Peter Relson has offered a free ‘as is, no
warranty’ tool to monitor program fetch activity: the Module Fetch
Monitor. Contact Peter directly at rel...@us.ibm.com.

On Wed, Nov 13, 2019 at 10:07 AM Steff Gladstone
 wrote:
>
> We would like to clean up our load libraries by deleting unused programs.
>
> Is there any way to use SMF data to track real-time usage of programs which
> are not main programs but are called by other programs?
>
> We were under the impression  that only executions of main programs (PGM=)
>  were recorded with SMF.  Scanning program sources for called programs is
> unsatisfactory for us since the call could be dynamic (name of program
> contained in a variable) or conditional on a particular set of
> circumstances that never occurs in practice.
>
> Thanks in advance!
>
> Steff Gladstone
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Scott Barry]

Consider solution P-Tracker (ESAi Group, developer: UBS-Hainer) for IBM z/OS 
software/application program invocation tracking for audit, compliance, 
asset-management.  Link: http://www.esaigroup.com/products/ptracker.htm

Scott Barry
SBBWorks, Inc.


On Wed, 13 Nov 2019 18:06:12 +0200, Steff Gladstone  
wrote:

>We would like to clean up our load libraries by deleting unused programs.
>
>Is there any way to use SMF data to track real-time usage of programs which
>are not main programs but are called by other programs?
>
>We were under the impression  that only executions of main programs (PGM=)
> were recorded with SMF.  Scanning program sources for called programs is
>unsatisfactory for us since the call could be dynamic (name of program
>contained in a variable) or conditional on a particular set of
>circumstances that never occurs in practice.
>
>Thanks in advance!
>
>Steff Gladstone
>
>--
>For IBM-MAIN subscribe / signoff / archive access instructions,
>send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Gord Tomlin]

On 2019-11-13 11:06, Steff Gladstone wrote:

We would like to clean up our load libraries by deleting unused programs.

Is there any way to use SMF data to track real-time usage of programs which
are not main programs but are called by other programs?

We were under the impression  that only executions of main programs (PGM=)
  were recorded with SMF.  Scanning program sources for called programs is
unsatisfactory for us since the call could be dynamic (name of program
contained in a variable) or conditional on a particular set of
circumstances that never occurs in practice.

Thanks in advance!

Steff Gladstone


Shameless plug: eventACTION has reference tracking features that can 
collect the information that you need for this cleanup.


--

Regards, Gord Tomlin
Action Software International
(a division of Mazda Computer Corporation)
Tel: (905) 470-7113, Fax: (905) 470-6507
Support: https://actionsoftware.com/support/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Salva Carrasco]

My two cents:

- Update every program to make a trace call. I'll prefer General tracking 
facility than Generic Trace.

- http://www.longpelaexpertise.com.au/ezine/SoftwareUsageWithoutTADz.php

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Seymour J Metz]

GTF


--
Shmuel (Seymour J.) Metz
http://mason.gmu.edu/~smetz3



From: IBM Mainframe Discussion List  on behalf of 
Steff Gladstone 
Sent: Wednesday, November 13, 2019 11:06 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Tracking called (non-main) programs using SMF records

We would like to clean up our load libraries by deleting unused programs.

Is there any way to use SMF data to track real-time usage of programs which
are not main programs but are called by other programs?

We were under the impression  that only executions of main programs (PGM=)
 were recorded with SMF.  Scanning program sources for called programs is
unsatisfactory for us since the call could be dynamic (name of program
contained in a variable) or conditional on a particular set of
circumstances that never occurs in practice.

Thanks in advance!

Steff Gladstone

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Leonardo Vaz]

If you are using ACF2 you can set LOG to the program on the PROGRAM class, I 
think RACF doesn’t log FASTAUTH requests but ACF2 does.

T R(PGM)
RECKEY myprog ADD(UID(*) LOG)

We actually use a homegrown ICHRTX00 exit to log program usage which works very 
well and it's "free" but like Charles said, it's not for the faint-hearted.

Regards,
Leo



-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Charles Mills
Sent: Wednesday, November 13, 2019 11:30 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: Tracking called (non-main) programs using SMF records

Similar topics have been kicked around here several times. You are basically 
correct -- SMF will not do the job you want it to do.

SMF 30 reports the "high CPU usage" program of the jobstep. That might be a 
different program than the jobstep program. It's an additional clue, but it is 
not the solution to your query.

I think the conclusion here was that there were some third-party products that 
do this. IIRC there is a CSV exit point that audits LOADs, but writing exits is 
not for the faint-hearted, and performance impact is a concern.

There is always the "you bet your job" approach. Move any questionable programs 
into a shadow copy of the load library and wait for the screams.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Steff Gladstone
Sent: Wednesday, November 13, 2019 8:06 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Tracking called (non-main) programs using SMF records

We would like to clean up our load libraries by deleting unused programs.

Is there any way to use SMF data to track real-time usage of programs which
are not main programs but are called by other programs?

We were under the impression  that only executions of main programs (PGM=)
 were recorded with SMF.  Scanning program sources for called programs is
unsatisfactory for us since the call could be dynamic (name of program
contained in a variable) or conditional on a particular set of
circumstances that never occurs in practice.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Tracking called (non-main) programs using SMF records

[Charles Mills]

Similar topics have been kicked around here several times. You are basically 
correct -- SMF will not do the job you want it to do.

SMF 30 reports the "high CPU usage" program of the jobstep. That might be a 
different program than the jobstep program. It's an additional clue, but it is 
not the solution to your query.

I think the conclusion here was that there were some third-party products that 
do this. IIRC there is a CSV exit point that audits LOADs, but writing exits is 
not for the faint-hearted, and performance impact is a concern.

There is always the "you bet your job" approach. Move any questionable programs 
into a shadow copy of the load library and wait for the screams.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Steff Gladstone
Sent: Wednesday, November 13, 2019 8:06 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Tracking called (non-main) programs using SMF records

We would like to clean up our load libraries by deleting unused programs.

Is there any way to use SMF data to track real-time usage of programs which
are not main programs but are called by other programs?

We were under the impression  that only executions of main programs (PGM=)
 were recorded with SMF.  Scanning program sources for called programs is
unsatisfactory for us since the call could be dynamic (name of program
contained in a variable) or conditional on a particular set of
circumstances that never occurs in practice.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Tracking called (non-main) programs using SMF records

[Steff Gladstone]

We would like to clean up our load libraries by deleting unused programs.

Is there any way to use SMF data to track real-time usage of programs which
are not main programs but are called by other programs?

We were under the impression  that only executions of main programs (PGM=)
 were recorded with SMF.  Scanning program sources for called programs is
unsatisfactory for us since the call could be dynamic (name of program
contained in a variable) or conditional on a particular set of
circumstances that never occurs in practice.

Thanks in advance!

Steff Gladstone

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sample SMF records

[Pierre Fichaud]

Gord,
I get the following when I click on z Technical Information:

 Information Central - IBM Z ISVs
Requested Information is Unavailable

I think I have to register with IBM.
I have sent an email to ztech.
Thanks, Pierre.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sample SMF records

[Gord Tomlin]

On 2019-10-27 09:00, Pierre Fichaud wrote:

Gord,
 I am logged into the secure area at https://dtsc.dfw.ibm.com/
 Where do I go next ?
Regards, Pierre.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN




1. Click "IBM Z® Technical Information".
2. Click "Optional Downloads".

--

Regards, Gord Tomlin
Action Software International
(a division of Mazda Computer Corporation)
Tel: (905) 470-7113, Fax: (905) 470-6507
Support: https://actionsoftware.com/support/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sample SMF records

[Pierre Fichaud]

Gord,
I am logged into the secure area at https://dtsc.dfw.ibm.com/
Where do I go next ?
Regards, Pierre.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Sample SMF records

[Gord Tomlin]

On 2019-10-26 14:14, Pierre Fichaud wrote:

Someone mentioned the PWD site has sample SMF data. I've logged into the site 
but can't find sample SMF records that I can look at or download. Can someone 
point me in the right direction please ?
Thanks in advance, Pierre.


I think you actually want the IBM Z Dallas ISV Center. Head over to
https://dtsc.dfw.ibm.com/MVSDS/'HTTPD1.ZS146.PUBLIC.HTML(DLOAD)'
on the secure portion of the site.

--

Regards, Gord Tomlin
Action Software International
(a division of Mazda Computer Corporation)
Tel: (905) 470-7113, Fax: (905) 470-6507
Support: https://actionsoftware.com/support/

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Sample SMF records

[Pierre Fichaud]

Someone mentioned the PWD site has sample SMF data. I've logged into the site 
but can't find sample SMF records that I can look at or download. Can someone 
point me in the right direction please ?
Thanks in advance, Pierre.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Charles Mills]

> I don't think IP addresses end up in the type 80s, though

They do not.

In really, really pursuing this I found IP addresses incredibly hard to audit 
anywhere downstream from the TN3270 mapping to an LU name.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Tony Harminc
Sent: Friday, May 10, 2019 5:58 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: LU name and RACF ID is SMF records

On Tue, 7 May 2019 at 09:04, Jorge Garcia  wrote:


>   We want to obtain LU name and RACF ID associated from SMF records or
> anyother source. We don't have available SMF record type 33. This LU name
> is available in TELNET profile
>
> LUGROUP LUMAJ
> T900D001..T900D030
>
> We don't know if there is some SMF records with the information of LU
> name, RACF ID and IP. We've executed test with SMF record 118 and 119 but
> we haven't obtain sucess.
>

I don't know the context - TSO sessions?

Some of the RACF type 80s contain an LU name, either as the POE part of the
RUTKN relocate section, or simply  in header field SMF80TRM. I don't think
IP addresses end up in the type 80s, though.

You may also find some TN3270 EZZ6034I messages in your Syslog that map LU
to IP:port, but these are normally only for error or forced logoff
situations unless you are running TN3270 tracing.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Tony Harminc]

On Tue, 7 May 2019 at 09:04, Jorge Garcia  wrote:


>   We want to obtain LU name and RACF ID associated from SMF records or
> anyother source. We don't have available SMF record type 33. This LU name
> is available in TELNET profile
>
> LUGROUP LUMAJ
> T900D001..T900D030
>
> We don't know if there is some SMF records with the information of LU
> name, RACF ID and IP. We've executed test with SMF record 118 and 119 but
> we haven't obtain sucess.
>

I don't know the context - TSO sessions?

Some of the RACF type 80s contain an LU name, either as the POE part of the
RUTKN relocate section, or simply  in header field SMF80TRM. I don't think
IP addresses end up in the type 80s, though.

You may also find some TN3270 EZZ6034I messages in your Syslog that map LU
to IP:port, but these are normally only for error or forced logoff
situations unless you are running TN3270 tracing.

Tony H.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Wolfgang Fritz]

If you don’t have smf119 records you can’t get information about lu‘s

Bin unterwegs hab nur iPhone zur Verfügung.

> Am 10.05.2019 um 09:54 schrieb Jorge Garcia :
> 
> Thanks Wolfgang, but we need an information in the past and we didnt' collect 
> this records at this time
> 
> Thanks
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Jorge Garcia]

Thanks Wolfgang, but we need an information in the past and we didnt' collect 
this records at this time

Thanks

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Wolfgang Fritz]

Hi Jorge
in  tcpip Profile add following statements to activate smf records in CS

SMFCONFIG; was SMFPARMS statementmnt
TYPE119
FTPCLIENT
IFSTATISTICS
IPSECURITY
PORTSTATISTICS
TCPINIT
TCPSTACK
TCPIPSTATISTICS
TCPTERM
TN3270CLIENT
UDPTERM

Am 08.05.2019 um 13:01 schrieb Wolfgang Fritz:

You should activate the smf119 in tcp/ip
Configure with your necessary subtypes

Bin unterwegs hab nur iPhone zur Verfügung.


Am 08.05.2019 um 09:26 schrieb Jorge Garcia :

Hi Wolgfang,

We don't have SMF records type 119 available in this system

Thanks

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Wolfgang Fritz]

Hi Jorge
you will find the smf defintion TCP/IP Books, use subtype 20 or 21 to 
get your information.


TN3270E Telnet server SNA session initiation record (subtype 20)
TN3270E Telnet server SNA session termination record (subtype 21)

if you need more help send me an mail on wolfgang.fr...@wfs-gmbh.eu
regards
Wolfgang
Am 08.05.2019 um 13:01 schrieb Wolfgang Fritz:

You should activate the smf119 in tcp/ip
Configure with your necessary subtypes

Bin unterwegs hab nur iPhone zur Verfügung.


Am 08.05.2019 um 09:26 schrieb Jorge Garcia :

Hi Wolgfang,

We don't have SMF records type 119 available in this system

Thanks

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Wolfgang Fritz]

You should activate the smf119 in tcp/ip
Configure with your necessary subtypes

Bin unterwegs hab nur iPhone zur Verfügung.

> Am 08.05.2019 um 09:26 schrieb Jorge Garcia :
> 
> Hi Wolgfang,
> 
> We don't have SMF records type 119 available in this system
> 
> Thanks
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Jorge Garcia]

Hi Wolgfang,

 We don't have SMF records type 119 available in this system

Thanks

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: LU name and RACF ID is SMF records

[Wolfgang Fritz]

Yes  you will find it in smf119 and you have to activate your Telnet records in 
tcpip



Bin unterwegs hab nur iPhone zur Verfügung.

> Am 07.05.2019 um 15:04 schrieb Jorge Garcia :
> 
> Hi all,
> 
> We want to obtain LU name and RACF ID associated from SMF records or anyother 
> source. We don't have available SMF record type 33. This LU name is available 
> in TELNET profile 
> 
> LUGROUP LUMAJ 
>T900D001..T900D030
> 
> We don't know if there is some SMF records with the information of LU name, 
> RACF ID and IP. We've executed test with SMF record 118 and 119 but we 
> haven't obtain sucess. 
> 
> Please, could you give us some advice to obtain this information?
> 
> Jorge Garcia Juanino
> Coordinador sistemas z/OS
> ACTP – DIAC – Operación y Soporte EMEA
> MAPFRE 
> Avenida del Talgo 100-103 – 3ª Planta
> CP 28023 Madrid
> Tel. 91 581 27 34, Movil 618333559 
> jgarc...@mapfre.com
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

LU name and RACF ID is SMF records

[Jorge Garcia]

Hi all,

 We want to obtain LU name and RACF ID associated from SMF records or anyother 
source. We don't have available SMF record type 33. This LU name is available 
in TELNET profile 

LUGROUP LUMAJ 
T900D001..T900D030

We don't know if there is some SMF records with the information of LU name, 
RACF ID and IP. We've executed test with SMF record 118 and 119 but we haven't 
obtain sucess. 

Please, could you give us some advice to obtain this information?

Jorge Garcia Juanino
Coordinador sistemas z/OS
ACTP – DIAC – Operación y Soporte EMEA
MAPFRE 
Avenida del Talgo 100-103 – 3ª Planta
CP 28023 Madrid
Tel. 91 581 27 34, Movil 618333559 
jgarc...@mapfre.com

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Question on SMF Records and ALTER GDG Base

[Beverly Caldwell]

Off hand I would try a 66. Have you looked in the SMF book?

On Sun, Aug 26, 2018 at 6:23 PM Lizette Koehler 
wrote:

> So, I have a request about who changed a GDG limit.
>
> Is there an SMF record that would list that type of change?
>
> I know the command will be ALTER gdg-base LIMIT(xx)
>
> What type of SMF Record would I look at?
>
> Thanks
>
>
> Lizette Koehler
> statistics: A precise and logical method for stating a half-truth
> inaccurately
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Question on SMF Records and ALTER GDG Base

[Lizette Koehler]

Len

Thanks, I was reviewing the Type 60's - Just did not go far enough

Lizette


> -Original Message-
> From: IBM Mainframe Discussion List  On Behalf Of
> Rugen, Len
> Sent: Sunday, August 26, 2018 6:28 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: Question on SMF Records and ALTER GDG Base
> 
> This, I think:
> 
> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.
> ieag200/iea3g2_Record_Type_66__42__Integrated_Catalog_Facility_Alter_Acti
> vity.htm
> 
> Len Rugen
> 
> University of Missouri
> Division of Information Technology
> Systems & Operations - Metrics & Automation Team
> 
> From: IBM Mainframe Discussion List  on behalf of
> Lizette Koehler 
> Sent: Sunday, August 26, 2018 8:23:05 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Question on SMF Records and ALTER GDG Base
> 
> So, I have a request about who changed a GDG limit.
> 
> Is there an SMF record that would list that type of change?
> 
> I know the command will be ALTER gdg-base LIMIT(xx)
> 
> What type of SMF Record would I look at?
> 
> Thanks
> 
> 
> Lizette Koehler
> statistics: A precise and logical method for stating a half-truth
> inaccurately
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send email to
> lists...@listserv.ua.edu with the message: INFO IBM-MAIN
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send email to
> lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: Question on SMF Records and ALTER GDG Base

[Rugen, Len]

This, I think:

https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.3.0/com.ibm.zos.v2r3.ieag200/iea3g2_Record_Type_66__42__Integrated_Catalog_Facility_Alter_Activity.htm

Len Rugen

University of Missouri
Division of Information Technology
Systems & Operations - Metrics & Automation Team

From: IBM Mainframe Discussion List  on behalf of 
Lizette Koehler 
Sent: Sunday, August 26, 2018 8:23:05 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Question on SMF Records and ALTER GDG Base

So, I have a request about who changed a GDG limit.

Is there an SMF record that would list that type of change?

I know the command will be ALTER gdg-base LIMIT(xx)

What type of SMF Record would I look at?

Thanks


Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Question on SMF Records and ALTER GDG Base

[Lizette Koehler]

So, I have a request about who changed a GDG limit.

Is there an SMF record that would list that type of change?

I know the command will be ALTER gdg-base LIMIT(xx) 

What type of SMF Record would I look at?

Thanks


Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Charles Mills]

@Kirk, my apologies. There are a couple of SFTP's out there and they are not 
all alike in their SMF approach. You guys did it right! (Don't re-invent the 
wheel.) My thanks.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Kirk Wolf
Sent: Wednesday, May 17, 2017 6:33 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SMF Records

FYI, Co:Z SFTP supports both Unix files and z/OS data sets and cuts SMF 119 
subtypes 3 and 70, the same as IBM FTP.

https://dovetail.com/docs/sftp/smf-support.html


Kirk Wolf
Dovetailed Technologies
http://dovetail.com

On Wed, May 17, 2017 at 8:18 AM, Charles Mills <charl...@mcn.org> wrote:

> 1. I think you are thinking of other subtypes, one each for SFTP 
> client (3) and server (70). Subtype 7 is the Server port statistics record:
> The Port Statistics record, as an interval record, periodically 
> records statistics on ports that have been configured with the PORT 
> statement in the TCP/IP PROFILE.
> All ports that were defined by the PORTRANGE statement, ports for 
> which the RESERVED flag has been set, or ports that were defined by 
> the PORT UNRSV statement are excluded.
>
> 2. But the OP is using SFTP, so "FTP" SMF records are irrelevant.
>
> Yes, you need to turn on SMF 119 in IP config if you want them.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Sasso, Leonard]

Sorry, I was not specific, I meant an MVS Dataset.

Many thanks to everyone, your responses have been very helpful.


Thank You,

Len Sasso
System Administrator


RDC - 327 Columbia TPKE, Rensselaer NY 12144-4400
t: +1.518.257.4209 | m: +1.518.894.0879
len.sa...@csra.com | www.csra.com Follow us on Facebook | Twitter | LinkedIn
CSRA
Think Next. Now.

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Kirk Wolf
Sent: Wednesday, May 17, 2017 9:33 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SMF Records

FYI, Co:Z SFTP supports both Unix files and z/OS data sets and cuts SMF 119 
subtypes 3 and 70, the same as IBM FTP.

https://dovetail.com/docs/sftp/smf-support.html


Kirk Wolf
Dovetailed Technologies
http://dovetail.com

On Wed, May 17, 2017 at 8:18 AM, Charles Mills <charl...@mcn.org> wrote:

> 1. I think you are thinking of other subtypes, one each for SFTP
> client (3) and server (70). Subtype 7 is the Server port statistics record:
> The Port Statistics record, as an interval record, periodically
> records statistics on ports that have been configured with the PORT
> statement in the TCP/IP PROFILE.
> All ports that were defined by the PORTRANGE statement, ports for
> which the RESERVED flag has been set, or ports that were defined by
> the PORT UNRSV statement are excluded.
>
> 2. But the OP is using SFTP, so "FTP" SMF records are irrelevant.
>
> Yes, you need to turn on SMF 119 in IP config if you want them.
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On Behalf Of Mike Myers
> Sent: Wednesday, May 17, 2017 3:18 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: SMF Records
>
> Leonard:
>
> I don't have access to the system any more where I did this, but based
> on the retained documents of the time, I gathered SMF type 119 records
> from TCP/IP. Subtype 7 records contain all data relating to FTP file 
> transfers.
> They give the file name, origin IP address, time stamp (date is
> relative and not in the record), receptor IP address, and file size.
> Filtering on these, I was able to find info on all files FTPed into or
> out of the z/OS box. You should be able to use this by sorting by file
> name to discover your over-writes. If I recall, it didn't matter
> whether the file was a USS or MVS file.
>
> As I recall, I did have to turn on SMF record creation in TCP/IP,
> which were not being generated by default.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions, send
> email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

This electronic message transmission contains information from CSRA that may be 
attorney-client privileged, proprietary or confidential. The information in 
this message is intended only for use by the individual(s) to whom it is 
addressed. If you believe you have received this message in error, please 
contact me immediately and be aware that any use, disclosure, copying or 
distribution of the contents of this message is strictly prohibited. NOTE: 
Regardless of content, this email shall not operate to bind CSRA to any order 
or other contract unless pursuant to explicit written agreement or government 
initiative expressly permitting the use of email for such purpose.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Kirk Wolf]

FYI, Co:Z SFTP supports both Unix files and z/OS data sets and cuts SMF 119
subtypes 3 and 70, the same as IBM FTP.

https://dovetail.com/docs/sftp/smf-support.html


Kirk Wolf
Dovetailed Technologies
http://dovetail.com

On Wed, May 17, 2017 at 8:18 AM, Charles Mills <charl...@mcn.org> wrote:

> 1. I think you are thinking of other subtypes, one each for SFTP client (3)
> and server (70). Subtype 7 is the Server port statistics record:
> The Port Statistics record, as an interval record, periodically records
> statistics on
> ports that have been configured with the PORT statement in the TCP/IP
> PROFILE.
> All ports that were defined by the PORTRANGE statement, ports for which the
> RESERVED flag has been set, or ports that were defined by the PORT UNRSV
> statement are excluded.
>
> 2. But the OP is using SFTP, so "FTP" SMF records are irrelevant.
>
> Yes, you need to turn on SMF 119 in IP config if you want them.
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Mike Myers
> Sent: Wednesday, May 17, 2017 3:18 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: SMF Records
>
> Leonard:
>
> I don't have access to the system any more where I did this, but based on
> the retained documents of the time, I gathered SMF type 119 records from
> TCP/IP. Subtype 7 records contain all data relating to FTP file transfers.
> They give the file name, origin IP address, time stamp (date is relative
> and
> not in the record), receptor IP address, and file size.
> Filtering on these, I was able to find info on all files FTPed into or out
> of the z/OS box. You should be able to use this by sorting by file name to
> discover your over-writes. If I recall, it didn't matter whether the file
> was a USS or MVS file.
>
> As I recall, I did have to turn on SMF record creation in TCP/IP, which
> were
> not being generated by default.
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Charles Mills]

1. I think you are thinking of other subtypes, one each for SFTP client (3)
and server (70). Subtype 7 is the Server port statistics record:
The Port Statistics record, as an interval record, periodically records
statistics on
ports that have been configured with the PORT statement in the TCP/IP
PROFILE.
All ports that were defined by the PORTRANGE statement, ports for which the
RESERVED flag has been set, or ports that were defined by the PORT UNRSV
statement are excluded.

2. But the OP is using SFTP, so "FTP" SMF records are irrelevant.

Yes, you need to turn on SMF 119 in IP config if you want them.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
Behalf Of Mike Myers
Sent: Wednesday, May 17, 2017 3:18 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: SMF Records

Leonard:

I don't have access to the system any more where I did this, but based on
the retained documents of the time, I gathered SMF type 119 records from
TCP/IP. Subtype 7 records contain all data relating to FTP file transfers.
They give the file name, origin IP address, time stamp (date is relative and
not in the record), receptor IP address, and file size. 
Filtering on these, I was able to find info on all files FTPed into or out
of the z/OS box. You should be able to use this by sorting by file name to
discover your over-writes. If I recall, it didn't matter whether the file
was a USS or MVS file.

As I recall, I did have to turn on SMF record creation in TCP/IP, which were
not being generated by default.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Mike Myers]

Leonard:

I don't have access to the system any more where I did this, but based 
on the retained documents of the time, I gathered SMF type 119 records 
from TCP/IP. Subtype 7 records contain all data relating to FTP file 
transfers. They give the file name, origin IP address, time stamp (date 
is relative and not in the record), receptor IP address, and file size. 
Filtering on these, I was able to find info on all files FTPed into or 
out of the z/OS box. You should be able to use this by sorting by file 
name to discover your over-writes. If I recall, it didn't matter whether 
the file was a USS or MVS file.


As I recall, I did have to turn on SMF record creation in TCP/IP, which 
were not being generated by default.


Mike Myers
Vice President
z/OS consultant
Mentor Services Corporation

 On 05/16/2017 05:47 PM, Sasso, Leonard wrote:

A customer transmits, via SFTP, a sequential file, from an external site to an 
IBM Mainframe. An hour later, they transmit the same file with the SAME file 
name, overwriting the original file.

1. What SMF Record Type was cut when the first file was transmitted?

2. Is another SMF Record cut for the overwrite and is it the SAME SMF Record 
Type as the original? If not, what SMF Record Type is cut?



Thank You,

Len Sasso
System Administrator
RDC - 327 Columbia TPKE, Rensselaer NY 12144-4400
t: +1.518.257.4209 | m: +1.518.894.0879
len.sa...@csra.com | www.csra.com Follow us on Facebook | Twitter | LinkedIn
CSRA
Think Next. Now.


This electronic message transmission contains information from CSRA that may be 
attorney-client privileged, proprietary or confidential. The information in 
this message is intended only for use by the individual(s) to whom it is 
addressed. If you believe you have received this message in error, please 
contact me immediately and be aware that any use, disclosure, copying or 
distribution of the contents of this message is strictly prohibited. NOTE: 
Regardless of content, this email shall not operate to bind CSRA to any order 
or other contract unless pursuant to explicit written agreement or government 
initiative expressly permitting the use of email for such purpose.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Vernooij, Kees (ITOPT1) - KLM]

> -Original Message-
> From: Vernooij, Kees (ITOPT1) - KLM
> Sent: 17 May, 2017 8:41
> To: 'IBM Mainframe Discussion List' <IBM-MAIN@LISTSERV.UA.EDU>
> Subject: RE: SMF Records
> 
> 
> 
> > -Original Message-
> > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> On
> > Behalf Of Jesse 1 Robinson
> > Sent: 17 May, 2017 0:18
> > To: IBM-MAIN@LISTSERV.UA.EDU
> > Subject: Re: SMF Records
> >
> > Keep in mind that SMF records are cut by applications or components
> that
> > choose to do so. It's not automatic. SMF is a record manager, not a
> > creator. In particular, SFTP is an add-on product that is not related
> to
> > FTP(S) or TCP/IP. It may or may not create SMF records. If it does,
> it's
> > almost certainly the same records type for every transmission.
> >
> > .
> > .
> 
> And in addition to that: look at the SMF manual: SMF 15 is written
> whenever a dataset is CLOSED after being opened for OUTPUT, period.
> Every time a dataset is Closed for output, SMF15 is written, whether it
> was the same dataset or not, the same user or not etc, just the CLOSE
> cuts the record. This allows you to monitor via SMF who opened a dataset
> during some perion, e.g. to determine the one that might have corrupted
> it.
> 
> Kees.

Additional to that: in SMFPRMxx you can specify which records SMF will write 
and which it will not write when the application cuts them and gives them to 
SMF tob e written.

Kees.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: SMF Records

[Vernooij, Kees (ITOPT1) - KLM]

> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Jesse 1 Robinson
> Sent: 17 May, 2017 0:18
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: SMF Records
> 
> Keep in mind that SMF records are cut by applications or components that
> choose to do so. It's not automatic. SMF is a record manager, not a
> creator. In particular, SFTP is an add-on product that is not related to
> FTP(S) or TCP/IP. It may or may not create SMF records. If it does, it's
> almost certainly the same records type for every transmission.
> 
> .
> .

And in addition to that: look at the SMF manual: SMF 15 is written whenever a 
dataset is CLOSED after being opened for OUTPUT, period. 
Every time a dataset is Closed for output, SMF15 is written, whether it was the 
same dataset or not, the same user or not etc, just the CLOSE cuts the record. 
This allows you to monitor via SMF who opened a dataset during some perion, 
e.g. to determine the one that might have corrupted it.

Kees.

For information, services and offers, please visit our web site: 
http://www.klm.com. This e-mail and any attachment may contain confidential and 
privileged material intended for the addressee only. If you are not the 
addressee, you are notified that no part of the e-mail or any attachment may be 
disclosed, copied or distributed, and that any other action related to this 
e-mail or attachment is strictly prohibited, and may be unlawful. If you have 
received this e-mail by error, please notify the sender immediately by return 
e-mail, and delete this message. 

Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its 
employees shall not be liable for the incorrect or incomplete transmission of 
this e-mail or any attachments, nor responsible for any delay in receipt. 
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch 
Airlines) is registered in Amstelveen, The Netherlands, with registered number 
33014286



--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN