Re: How to speed up IFASMFDP SMF Dump process

[Richards, Robert B.]

Ours range from 300 to 700 cylinders per type (Default, RMF, DB2, RACF)

DFHSM handles the migration of them when they fill up.

Bob

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Elardus Engelbrecht
Sent: Thursday, March 15, 2018 9:29 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: How to speed up IFASMFDP SMF Dump process

Radoslaw Skorupka wrote:

>> I don't know about IFASMFDL
>AFAIK the IFASMFDL is "system logger mutation" of IFASMFDP. 

Thanks. I just forgot about this explanation.


>Note, currently IBM pushes customers to use logger instead of SYS1.MAN 
>datasets.

I am just waiting for big blue's SOD that SYS1.MANx and IFASMFDP are to be 
dropped, then I will hurry up... ;-)


I know there is a general sizer where you can estimate your structures' sizes, 
but is there a sizer for SMF logger(s) in XCF where you can 'guess' how big it 
should be based on x million records in y hours from z LPARs?

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Elardus Engelbrecht]

Radoslaw Skorupka wrote:

>> I don't know about IFASMFDL
>AFAIK the IFASMFDL is "system logger mutation" of IFASMFDP. 

Thanks. I just forgot about this explanation.


>Note, currently IBM pushes customers to use logger instead of SYS1.MAN 
>datasets.

I am just waiting for big blue's SOD that SYS1.MANx and IFASMFDP are to be 
dropped, then I will hurry up... ;-)


I know there is a general sizer where you can estimate your structures' sizes, 
but is there a sizer for SMF logger(s) in XCF where you can 'guess' how big it 
should be based on x million records in y hours from z LPARs?

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[R.S.]

W dniu 2018-03-15 o 10:14, Elardus Engelbrecht pisze:

Martin Packer wrote:


I always thought IFASMFDP (and now IFASMFDL) could do with an exit point - so 
you could reformat (flatten, essentially) or filter records using it. That 
might cut down data moves.

IFASMFDP has such exit points where USER1/2/3 exit points are given control 
after each record is read and before that record is written out or not 
depending on exits own RC=0/4.

See this IRRADU00 example which 'reformat and flatten' the selected records [ 
for RACF unloads].

//SMF   EXEC PGM=IFASMFDP
//INDD DD DISP=SHR,DSN=
//OUTDDDD DISP=OLD,DSN=IRRADU00
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(200,100))
//SYSPRINT DD SYSOUT=*
//ADUPRINT DD SYSOUT=*
//SMFOUT   DD DUMMY
//SYSINDD *
INDD(INDD,OPTIONS(DUMP))
OUTDD(SMFOUT,TYPE(20,30,80,81,83))
ABEND(NORETRY)
USER2(IRRADU00)
USER3(IRRADU86)

I don't know about IFASMFDL


AFAIK the IFASMFDL is "system logger mutation" of IFASMFDP. Note, 
currently IBM pushes customers to use logger instead of SYS1.MAN datasets.


Regards
--
Radoslaw Skorupka
Lodz, Poland




==


   --
Treść tej wiadomości może zawierać informacje prawnie chronione Banku 
przeznaczone wyłącznie do użytku służbowego adresata. Odbiorcą może być jedynie 
jej adresat z wyłączeniem dostępu osób trzecich. Jeżeli nie jesteś adresatem 
niniejszej wiadomości lub pracownikiem upoważnionym do jej przekazania 
adresatowi, informujemy, że jej rozpowszechnianie, kopiowanie, rozprowadzanie 
lub inne działanie o podobnym charakterze jest prawnie zabronione i może być 
karalne. Jeżeli otrzymałeś tę wiadomość omyłkowo, prosimy niezwłocznie 
zawiadomić nadawcę wysyłając odpowiedź oraz trwale usunąć tę wiadomość 
włączając w to wszelkie jej kopie wydrukowane lub zapisane na dysku.

This e-mail may contain legally privileged information of the Bank and is 
intended solely for business use of the addressee. This e-mail may only be 
received by the addressee and may not be disclosed to any third parties. If you 
are not the intended addressee of this e-mail or the employee authorized to 
forward it to the addressee, be advised that any dissemination, copying, 
distribution or any other similar activity is legally prohibited and may be 
punishable. If you received this e-mail by mistake please advise the sender 
immediately by using the reply facility in your e-mail software and delete 
permanently this e-mail including any copies of it either printed or saved to 
hard drive.

mBank S.A. z siedzibą w Warszawie, ul. Senatorska 18, 00-950 Warszawa, 
www.mBank.pl, e-mail: kont...@mbank.plsąd Rejonowy dla m. st. Warszawy XII 
Wydział Gospodarczy Krajowego Rejestru Sądowego, nr rejestru przedsiębiorców 
KRS 025237, NIP: 526-021-50-88. Według stanu na dzień 01.01.2018 r. kapitał 
zakładowy mBanku S.A. (w całości wpłacony) wynosi 169.248.488 złotych.
   


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Elardus Engelbrecht]

Martin Packer wrote:

>I always thought IFASMFDP (and now IFASMFDL) could do with an exit point - so 
>you could reformat (flatten, essentially) or filter records using it. That 
>might cut down data moves.

IFASMFDP has such exit points where USER1/2/3 exit points are given control 
after each record is read and before that record is written out or not 
depending on exits own RC=0/4. 

See this IRRADU00 example which 'reformat and flatten' the selected records [ 
for RACF unloads].

//SMF   EXEC PGM=IFASMFDP  
//INDD DD DISP=SHR,DSN=
//OUTDDDD DISP=OLD,DSN=IRRADU00
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(200,100))
//SYSPRINT DD SYSOUT=*
//ADUPRINT DD SYSOUT=*
//SMFOUT   DD DUMMY   
//SYSINDD *   
INDD(INDD,OPTIONS(DUMP))  
OUTDD(SMFOUT,TYPE(20,30,80,81,83))
ABEND(NORETRY)
USER2(IRRADU00)   
USER3(IRRADU86)   

I don't know about IFASMFDL

>Next time I talk to “Mr SMF” I should ask him - but I’m already tying up 
>enough of his time. :-)

;-D

Groete /  Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Martin Packer]

I always thought IFASMFDP (and now IFASMFDL) could do with an exit point -
so you could reformat (flatten, essentially) or filter records using it.
That might cut down data moves.

Next time I talk to “Mr SMF” I should ask him - but I’m already tying up
enough of his time. :-)

Cheers, Martin

Sent from my iPad

> On 14 Mar 2018, at 17:52, William Richardson  wrote:
>
> One thing to consider is the number, type and complexity of the control
cards being used in the dump utility request itself;
> if you are using a "lot" of output targets you might consider using a
"SORT" type program instead of the IFASMFDP utility itself for the dump
process (you still need it for the CLEAR).
>
> "SORT" programs (in general) are much more efficient in doing this type
of I/O processing; potentially multi-task or multi-threading the work.
>
> Of course the down side of this is having to translate the SMF defined
control card "terms" into offsets (ie. 'Type' is at offset "x" and is of
length "1") and taking into account the RDW;  the good thing would be this
conversion is probably already in use in other jobs that are sorting SMF
data.
>
>
> NOTE: This is just MY personal observation based on my knowledge of the
processing and NOT a formal recommendation by the current SMF team  or
anything official.
>
> Thanks,
> Bill..
> z/OS System Software Development
> (former SMF Component Owner)
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[William Richardson]

One thing to consider is the number, type and complexity of the control cards 
being used in the dump utility request itself;
if you are using a "lot" of output targets you might consider using a "SORT" 
type program instead of the IFASMFDP utility itself for the dump process (you 
still need it for the CLEAR).

"SORT" programs (in general) are much more efficient in doing this type of I/O 
processing; potentially multi-task or multi-threading the work.

Of course the down side of this is having to translate the SMF defined control 
card "terms" into offsets (ie. 'Type' is at offset "x" and is of length "1") 
and taking into account the RDW;  the good thing would be this conversion is 
probably already in use in other jobs that are sorting SMF data.


NOTE: This is just MY personal observation based on my knowledge of the 
processing and NOT a formal recommendation by the current SMF team  or anything 
official.

Thanks,
Bill..
z/OS System Software Development
(former SMF Component Owner)

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

RES: How to speed up IFASMFDP SMF Dump process

[ITURIEL DO NASCIMENTO NETO]

Hello,

One possible action is to enhance BUFSPACE of SYS1.MAN* to 90112.
I think it can be done via ALTER command.

Atenciosamente / Regards / Saludos


Ituriel do Nascimento Neto
BANCO BRADESCO S.A.
4250 / DITI Engenharia de Software
Sistemas Operacionais Mainframes
Tel: +55 11 3684-9602 R: 49602 3-1404
Fax: +55 11 3684-4427



-Mensagem original-
De: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] Em nome de 
Lizette Koehler
Enviada em: segunda-feira, 12 de março de 2018 16:38
Para: IBM-MAIN@LISTSERV.UA.EDU
Assunto: How to speed up IFASMFDP SMF Dump process

I was wondering if there are any tricks to making the SMF Dump process more 
efficient or faster?

And parms on the VSAM Define for the MANx files (like STRNO or RLS) that might 
help?

I know about the SMF LOGSTREAM - but not ready to do that yet


I am just using the IBM program IFASMFDP for now



Thanks



Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

--
For IBM-MAIN subscribe / signoff / archive access instructions, send email to 
lists...@listserv.ua.edu with the message: INFO IBM-MAIN

AVISO LEGAL ...Esta mensagem é destinada exclusivamente para a(s) pessoa(s) 
a quem é dirigida, podendo conter informação confidencial e/ou legalmente 
privilegiada. Se você não for destinatário desta mensagem, desde já fica 
notificado de abster-se a divulgar, copiar, distribuir, examinar ou, de 
qualquer forma, utilizar a informação contida nesta mensagem, por ser ilegal. 
Caso você tenha recebido esta mensagem por engano, pedimos que nos retorne este 
E-Mail, promovendo, desde logo, a eliminação do seu conteúdo em sua base de 
dados, registros ou sistema de controle. Fica desprovida de eficácia e validade 
a mensagem que contiver vínculos obrigacionais, expedida por quem não detenha 
poderes de representação. 
LEGAL ADVICE...This message is exclusively destined for the people to whom 
it is directed, and it can bear private and/or legally exceptional information. 
If you are not addressee of this message, since now you are advised to not 
release, copy, distribute, check or, otherwise, use the information contained 
in this message, because it is illegal. If you received this message by 
mistake, we ask you to return this email, making possible, as soon as possible, 
the elimination of its contents of your database, registrations or controls 
system. The message that bears any mandatory links, issued by someone who has 
no representation powers, shall be null or void.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[William Richardson]

>>Set MANx CISIZE to half track (26624).  
> Interesting. Why half track? Is it documented that it will help? Just curious 
> if you don't mind, please.

Short answer ... 26K (or even 16K) of data written per I/O is much better 
than the default of 4K (CI Size).
It really speeds up the writes to the MANx dataset done by the SMF "writer".
(HALF Track is the best in terms of "head movement" in spinning DASD)

and Yes the down side is that the SMF writer does NOT include proper post 
record "spanning" (or packing).

Bill .
IBM z/OS System Software Development
(and former SMF Component owner at the time the above was done)


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Tim Hare]

Let me add to my post - if you read abut SMF CI Size in member ACHAP03 of the 
MXG  source/documentation library, you will see a very good discussion of the 
trade-off between DASD space and CISIZE in MANx datasets.  Apparently SMF 
doesn't do VBS in the MANx datasets in the same way "regular" VBS is done, and 
can waste space because of how it does it.  The ANALSMF program includes what 
they call an "SMF Simulator" that attempts to help you find the optimum CISIZE 
for the SMF records you produce.

There are also a lot of other SMF tips in ACHAP03 that are worth reading.  If 
you don't have MXG perhaps you could contact Dr. Merrill and ask if he would 
share that information as a separate document or if he has done a SHARE 
presentation about it.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Elardus Engelbrecht]

Tim Hare wrote:

>Set MANx CISIZE to half track (26624).  

Interesting. Why half track? Is it documented that it will help? Just curious 
if you don't mind, please.


>//AMP='BUFND=60',
>60 buffers = 2 cylinders if you are using half track CI sizes.

That is a good one. I really need to investigate that AMP and its friends.


>On the output sequential datasets on disk we use 
>RECFM=VBS,LRECL=32760,BLKSIZE=0  and let the system choose the blocking. 

Oh, yes, I should have mentioned that. This is what I always do. Let the sytem 
decide about blocksize. Thanks Tim.


>If you have huge amounts of data you might consider using striping for the 
>output if it is going to disk;

Striping? Hm. Ok, I will really need to talk with my storage admin about 
this little gem.

Thanks Tim. You burned a hole in my dizzy brain. ;-)

I really appreciate your kind and helpful reply to Lizette's post. Much 
appreciated.

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Tim Hare]

Do you have MXG?  They have a program, ANALSMF, which will analyze your current 
SMF.  In general, what I've done and which seemed to help:

Set MANx CISIZE to half track (26624).  

In your IFASMFDP JCL to dump a full MANx dataset (we use IEFU29 to issue a 
start for SMFDUMP JCL to dump one whenever it is full, your method may vary),  
use this for the input, so  represents the MANx dataset name.

//SMF  DD DISP=SHR,  
//AMP='BUFND=60',
 //   DSN= 

60 buffers = 2 cylinders if you are using half track CI sizes.

On the output sequential datasets on disk we use 
RECFM=VBS,LRECL=32760,BLKSIZE=0  and let the system choose the blocking. The 
last time we did this we had SMS turn on compression for these so it shows 
BLKSIZE=32760 but I believe if you don't use compression the system will choose 
half-track blocking.  Keep in mind that compression uses CPU time unless you 
have zEDC.

The important part: we use BUFNO=60 to match the input.  We never tested 
whether output needed to be larger than input, or not.

If you have huge amounts of data you might consider using striping for the 
output if it is going to disk;  I don't know how much it will help elapsed time 
for the dump but it will help the elapsed time of  whatever uses the data.  I 
tested it once but we didn't implement it.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: How to speed up IFASMFDP SMF Dump process

[Elardus Engelbrecht]

Lizette Koehler wrote:

>I was wondering if there are any tricks to making the SMF Dump process more 
>efficient or faster?

I have gotten many problems with the SMF dump process - anything from slow 
dumping, ENQs accross SysPlex and also during IPL, Timing issues, running out 
of space, name it. I have gotten all of them and mentioned here on IBM-MAIN...


>And parms on the VSAM Define for the MANx files (like STRNO or RLS) that might 
>help?

I don't know what are the best values, but if someone has good suggestions plus 
others, I'm all ears! 


>I am just using the IBM program IFASMFDP for now

Ok, below actions resolved all problems including the ENQ and IOS120D during 
IPLs.

All VSAM datasets in the SysPlex are on exclusive Non-SMS volsers. Nothing else 
are there. This alone reduce fragmentation.

All my other SMF datasets are in exclusive SMS management/dataclass/storage 
group/storage class.

These SMFPRMxx settings resolved many other dump timing problems:

BUFUSEWARN(25 or 50 depending on LPAR)   
BUFSIZMAX(512M)  (Larger is better. See above)
MAXDORM(0500)  (Why filling it up?)
DDCONS(NO) 

Depending on speed of filling up, I want to have automation to kick off 'I SMF' 
automagically as few times as possible. Let z/OS do it (dump/clear VSAM dsn) 
instead forcing it.

The idea is - dump more often in smaller chunks.

Then I have activated custom IEFU29 process which are kicked off automatically 
whenever there is a 'I SMF'.

Another tailoring I did was having every 3 - 6 hours a job which forced a 'I 
SMF' about one minute apart for all LPARs in the SysPlex.

Setup your automation software to handle IEE366*, *IEE986E, *IEE985A, IEE979* 
messages.

After midnight, everything written out are then copied and MODded on a monthly 
dataset. Repeat all actions for having a working backup.

Of course, my SMF setup may or may not work in YOUR environment - YMMV. No 
LOGSTREAM for me for now.

Hope that helps you!

Groete / Greetings
Elardus Engelbrecht

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

How to speed up IFASMFDP SMF Dump process

[Lizette Koehler]

I was wondering if there are any tricks to making the SMF Dump process more
efficient or faster?

And parms on the VSAM Define for the MANx files (like STRNO or RLS) that might
help?

I know about the SMF LOGSTREAM - but not ready to do that yet


I am just using the IBM program IFASMFDP for now



Thanks



Lizette Koehler
statistics: A precise and logical method for stating a half-truth inaccurately

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN